Altimeter's Agrawal Says Chip Dominance Squeezes AI App Margins

Altimeter的Agrawal称芯片主导地位挤压AI应用利润率 在高昂的推理成本下降以及各平台采用广告模式实现免费用户变现之前，AI生态系统仍将是一个严重偏向底层的金字塔结构。 讲师Apoorv Agrawal将当前AI技术的经济模型与以往的技术周期进行了对比，强调价值仍高度集中在底层基础设施和半导体层，而非流向应用层。他分析了NVIDIA强劲的毛利率，以及AI应用面临的可变运行成本与微薄利润，并将这些动态特征与传统软件进行对照。本次讲座探讨了定制芯片领域的客户集中度、垂直整合的历史模式，以及ChatGPT等消费级AI平台的规模局限。最后，Agrawal指出，消费级AI应用最终必须采用高意图对话式广告模式，以实现长期盈利。 课程概述与安排：MS&E435探讨了AI超级周期的经济结构，设有遵循查塔姆研究所规则（Chatham House Rules）的嘉宾演讲，每周课业负担较轻（约3小时），评分标准为出勤占50%、期末作业占50%。 (0:09) 构建AI超级周期框架：AI代表着一场前所未有的经济转型，学生未来很可能会创立或投资下一代AI公司，这需要清晰的心智模型来评估各种机遇。 (4:00) 正三角与倒金字塔：与以往价值集中在应用层的技术浪潮（云计算、移动互联网、Web）不同，目前的AI生态系统呈现为一个正三角形，价值高度集中在底层的芯片与基础设施。 (5:37) AI应用经济学：传统软件由于边际分发成本接近于零而拥有80-90%的毛利率，而AI推理过程中持续的GPU使用产生了高额的可变运行成本，将应用利润率压缩至0-30%。 (7:15) 基础设施时间线：物理基础设施的建设周期需要相当长的时间，顶层应用收入才能跟上步伐，AWS从推出到完全实现内部迁移所经历的八年酝酿期便说明了这一点。 (9:20)

Instructor Apoorv Agrawal argues that the generative AI ecosystem forms an upright economic pyramid dominated by semiconductor profits, contrasting with prior tech waves where value accumulated in high-margin applications. Because inference imposes continuous compute costs while consumer monetization remains low, sustained market rebalancing will require hyperscaler silicon competition and conversational advertising models. The Upright Value Triangle and Margin Compression (5:37) In historical internet, mobile, and cloud supercycles, the computing stack resembled an inverted pyramid where the application layer captured the vast majority of total economic value. Traditional software businesses enjoyed zero marginal distribution costs, generating 80% to 90% gross margins. By contrast, Agrawal describes the generative AI landscape as an upright triangle where capital and profits are heavily concentrated at the base in semiconductors and physical infrastructure, while application software generates comparatively small returns. This inversion stems from the unit economics of inference. Every incremental AI query requires continuous GPU cycles, saddling application developers with high variable operating costs and compressing gross margins to between 0% and 30%. Over the last two years, although the AI ecosystem added approximately $350 billion in total revenue and application revenue grew more than tenfold, roughly 75% of that new revenue flowed directly to the semiconductor layer—dominated almost entirely by Nvidia, which maintains data center gross margins near 75%. Workload Physics and Hyperscaler Silicon Concentration (21:07) Operational workloads in generative AI are split between model training and inference. Nvidia disclosures indicate that roughly 60% of deployed GPUs are utilized for training, which requires predictable and continuous high utilization, while 40% run inference workloads that fluctuate according to daytime user activity. Agrawal expects inference to expand as autonomous AI agents take over and establish continuous 24/7 baselines. Hardware revenue remains narrow and concentrated. As Nvidia discloses, approximately half of its market revenue originates directly from major hyperscalers like AWS, Google, and Microsoft. For emerging ASIC startups, the customer base is limited to a small group of massive buyers rather than an enterprise funnel. Agrawal highlights two potential catalysts that could disrupt this concentration: breakout success from internal hyperscaler silicon programs (including Google TPU, Meta MTIA, and internal chips from Amazon and Microsoft) or a public scale-back in hyperscaler infrastructure CapEx guidance. Consumer Adoption Limits and Monetization Disparities (28:37) Consumer AI applications face both an engagement ceiling and a severe monetization gap. Agrawal categorizes legacy consumer platforms into three scale tiers: mandatory utilities with over 3 billion users (WhatsApp, Chrome), social network applications with 1.5 to 2 billion users (Instagram, TikTok), and niche or task-specific destinations below 1.5 billion users (Amazon, Spotify). ChatGPT has only recently moved beyond the niche tier toward social scale, while Google Gemini remains below it. Agrawal attributes this adoption constraint to the friction of interaction: unlike passive social media feeds, conversational AI demands active intellectual labor from users. Consequently, monetization remains weak, with roughly 95% of ChatGPT users on the free tier. OpenAI currently generates approximately $10 per user annually across roughly 1 billion users, compared to roughly $70 per user for Meta across 3.5 billion users, and roughly $100 per user for Alphabet across 4 billion users. The Strategic Shift Toward Conversational Advertising (33:02) To close the average revenue per user gap between AI services and established consumer giants, Agrawal contends that subscription revenue alone will prove insufficient. Drawing a parallel to Wall Street's skepticism during Facebook's 2012 transition to mobile interfaces, he projects that generative AI platforms will inevitably turn to advertising to monetize their broad, unpaid user bases. Agrawal predicts that conversational advertising formats will command higher pricing (CPMs) than legacy search or display ads. Conversational environments benefit from high user trust, authenticated log-in states, direct attribution, and deep visibility into user intent. Despite current design challenges and consumer hesitation surrounding ad placements within intimate chat interfaces, Agrawal maintains that developing a native conversational ad format represents the critical financial unlock for consumer AI sustainability. Whether the generative AI economic pyramid will eventually invert into a software-centric model—and whether that shift will take five, fifteen years, or never materialize—remains unresolved. Agrawal emphasizes that the long-term equilibrium of the supercycle hinges on whether hyperscalers can lower compute costs via custom silicon while application providers crack conversational ad monetization.

在斯坦福大学MS&E435课程中，主讲人兼Altimeter Capital投资人Apoorv Agrawal提出，与以往底层硬件薄利、应用层捕获主要价值的软件周期不同，当前生成式AI生态呈现出利润极度集中于芯片和基础设施底层的“正金字塔”形态。Agrawal认为，AI应用面临持续的推理计算成本与薄弱的单用户变现能力，未来行业若要走向健康的经济结构，不仅依赖巨头自研芯片平抑硬件溢价，更取决于消费级平台能否成功探索出对话式广告变现模式。 正金字塔结构与软件边际成本分歧 (5:37) Agrawal回顾了过去25年的互联网、20年前的移动浪潮及10年前的云计算周期，指出历史技术生态最终均演变为应用层贡献绝大部分经济价值的“倒金字塔”结构。传统软件行业之所以能“吞噬世界”，核心在于其边际分发成本趋近于零，普遍享有80%至90%的高毛利率。然而，当前生成式AI生态却呈现相反的“正金字塔”形态：超大规模云厂商和半导体企业向包含能源、芯片、电力、互连及存储的数据中心“五层架构”注入巨额资本开支，大量价值被固化在底层的算力租赁与基础设施环节。 造成这一分歧的关键在于AI应用底层的运行规律与传统软件截然不同。用户对AI模型的每次调用都会在推理阶段持续占用GPU计算周期，产生高昂的可变运行成本，导致AI应用即使达到庞大营收规模也难以实现可观利润。Agrawal援引AWS从2004年破土动工、2010年签约Netflix至2012年完成Amazon自身业务全面迁移的历史进程指出，物理硬件基础设施的成熟往往需要长达八年以上的周期，应用层收入对底层硬件投资的追赶与释放难以在短期内一蹴而就。 硬件高额利润与巨头自研芯片博弈 (19:13) 根据Agrawal展示的生态价值分配数据，在过去两年中，AI生态系统总体规模扩张约5倍，新增约3500亿美元收入，但其中约75%的新增收入直接沉淀至以NVIDIA为主的半导体层。产业链各层级的盈利水平分化严重：NVIDIA数据中心业务的毛利率高达约75%，而应用层企业的毛利率仅在0%至30%的极低区间徘徊，且超过90%的应用收入被前两家领头企业垄断。中间的基础设施层则因云巨头自研与初创企业激烈竞争，成为更替率和竞争烈度最高、市场格局最不稳定的环节。 在硬件需求侧，市场约3000亿美元的芯片收入中，约半数直接由少数几家超大规模云厂商贡献。这种高度集中的买方结构意味着初创芯片企业面临极窄的客户群体，必须以打入少数云巨头供应链为核心目标。Agrawal分析指出，若要打破当前半导体独占高利润的格局，关键催化剂在于两点：一是Google TPU、Meta MTIA及亚马逊、微软等云厂商内部定制ASIC项目的突破；二是公开财报中超大规模云厂商资本开支增速的放缓，后者将倒逼硬件基础设施重新评估投资回报。 算力负载分化与全栈垂直整合规律 (21:07) 在算力运行特征上，模型训练与推理呈现截然不同的经济曲线。训练负载具有高度确定性，通常在固定周期内维持连续且极高的算力利用率；推理负载则极具波动性，紧随人类作息呈现明显的日间高峰与节假日低谷。Agrawal引用NVIDIA财报说明，当前其部署的GPU中约40%用于推理，60%用于训练，预计未来推理占比将逐步扩大，且长远来看自主AI智能体的普及有望将推理负载拉平为全天候持续运行模式。 纵观科技演进历史，Agrawal认为最终的超级赢家通常具备深度垂直整合能力：Web时代的Google从自研服务器到搜索索引及广告拍卖实现了端到端控制；移动时代的Apple掌控了自研芯片、硬件系统及应用生态；云计算时代则由AWS、Azure和GCP形成寡头竞争格局。在当下的AI周期中，NVIDIA正通过DGX Cloud和行业软件主动向产业链上层渗透，而专注于单一推理或中间件的独立初创企业，则面临其产品形态被云服务商直接整合为标准内建功能的严峻风险。 消费级规模瓶颈与对话广告变现探索 (28:37) 在消费级AI领域，应用扩张面临显著的用户规模瓶颈与单客变现短板。Agrawal将传统互联网产品划分为超30亿用户的“强制性生活基础设施”（如WhatsApp、Chrome）、15亿至20亿用户的“社交网络效应平台”（如Instagram、TikTok）及15亿以下聚焦特定用途的工具类平台（如Spotify、Amazon）。目前ChatGPT刚脱离小众工具步入社交平台级别的用户规模，Gemini位列第二，但均未成为全民级生活工具。Agrawal解释称，不同于被动消费内容的社交媒体，对话式AI需要用户投入主动智力进行交互，天然筛选了用户群体。 这种使用特质导致了严重的商业化落差：Google年均每用户贡献约100美元营收，Meta约为70美元，而拥有约10亿级用户的ChatGPT单客年收入仅约10美元，且约95%的用户停留在免费版本。Google DeepMind管理层亦公开表示不会单纯依靠订阅模式。Agrawal对比了2012年Facebook上市时外界对其移动端屏幕无法展示广告的质疑，预测未来AI平台势必走向对话式广告；对话界面拥有极强的用户意图、登录状态与信任度，若能跑通适配对话场景的原生广告格式，将成为扭转AI应用经济效益的关键突破口。 Agrawal总结指出，当前生成式AI的价值捕获正金字塔能否在未来5至15年内回归传统云计算的倒金字塔形态，仍是一个悬而未决的核心议题。这不仅取决于推理成本的下降与巨头定制芯片的替代效应，还取决于消费级应用能否在保持用户信任的前提下成功开创对话式广告等高效商业化途径，从而改变目前应用层毛利微薄、硬件层独占利润的失衡格局。

[4:00] Apoorv Agrawal: laws of physics [4:00] Apoorv Agrawal: the tectonic plates were forming and clay was forming [5:37] Jensen Huang: 5-layer cake [5:37] Apoorv Agrawal: ate the world [19:13] Apoorv Agrawal: bursty [21:07] Apoorv Agrawal: Which of the five hyperscalers are you going to sell to first? [24:07] Apoorv Agrawal: the fastest way to be fast is to do fast inference and drop that thing into Claude. [24:07] Apoorv Agrawal: Just give the human players five seconds, let them go at it and let them win the analog way. [25:51] Apoorv Agrawal: metabolic rate [28:37] Apoorv Agrawal: It's a place where you go and you have to do active work... And the number of people in the world who are asking active questions of technology is not the entirety of the population that's online. [33:02] Apoorv Agrawal: Shocker, we found the space on a phone.

[0:09] All right, folks, we're going to have some fun in this session. [0:15] My name is Apoorva. [0:16] I'm going to be your instructor for the next nine weeks or so. [0:21] Here's what we're going to go through today. [0:23] I'm going to talk a little bit about myself. [0:25] Why do I do this. [0:27] Some logistics on what to expect. [0:31] Quiz. [0:32] Yep, I'm that guy. [0:33] We're going to have quiz on day one. [0:36] And the biggest question that we've all been wrestling with, [0:40] where's the money? [0:41] Where's the money in AI? [0:42] Some of you know me, but my journey started in India. [0:47] I moved to Singapore. [0:48] I met a couple of Singaporean folks here earlier. [0:50] I started my career at Palantir with Sunil [0:54] and a couple of other folks about 13 years ago, [0:56] 14 years ago. [0:57] Led a variety of engineering teams. [1:00] All that to say, we wrote a lot of spark [1:02] in government buildings. [1:03] And I came back to Stanford for grad school, [1:06] which is when I got tired of writing [1:08] Spark in government buildings. [1:10] And now, I lead Altimeter. [1:12] I don't know how many of you have heard of Altimeter, [1:14] but Altimeter is an investment firm. [1:15] We focus on fairly concentrated form of investing. [1:19] We've got two businesses. [1:20] We've got a public business and a private business. [1:22] And then I got the biggest promotion of my life [1:25] six months ago. [1:26] I'm now the proud dad. [1:31] This is, as people have told me, the biggest investment [1:35] I will make. [1:36] Some have called it the one with the most guaranteed [1:39] negative IRR. [1:41] I think it's the most guaranteed positive IRR, not financial, [1:46] but that's me. [1:46] I live across the street. [1:49] Reach out with questions. [1:50] I want to be here as a resource to you guys. [1:53] I'm joined by an incredible TA in Chloe Feng. [1:57] Reach out to her if you have harder questions-- [1:59] [APPLAUSE] [2:03] - And make the most of it. [2:06] We've got a great session lined up for you guys. [2:08] The course is designed to be no more than three hours a week, [2:12] and that includes class readings all the time [2:15] you spend arguing with ChatGPT, with Claude about whether it [2:19] can do your assignment for you. [2:21] So it's about an hour of class. [2:24] It's about an hour or two of readings. [2:26] And basically, the format is we're [2:29] going to do guest speakers every single class, next class [2:32] onwards. [2:34] Chatham House Rules a lot of guest speakers [2:36] will share, maybe overshare. [2:39] So please don't record what they're saying. [2:44] We'll have an optional dinner with some of them right [2:46] afterwards. [2:47] You guys are welcome to join. [2:49] Chloe will arrange the logistics. [2:53] Grading is easy. [2:54] 50/50, show up to class. [2:56] If Ali Ghodsi can show up to class, you can show up to class. [2:59] And the other half is an assignment [3:01] that we will release at the end of the course. [3:06] Yeah, it's conversational. [3:07] Ask questions, be involved. [3:08] The more you're involved, the you're going to get out of it. [3:11] And honestly, what's in it for me [3:13] is I'm going to learn the most from you guys. [3:15] This is the course schedule over the next nine weeks. [3:18] Lots of great speakers, one more impressive than the other. [3:21] As for me, honestly, putting this calendar together [3:23] has been my job for the last couple of weeks and months. [3:28] Chloe knows all about it. [3:29] But be present. [3:30] These are all incredible leaders running incredible businesses [3:33] across the stack from Semis to Infra [3:37] to energy on the infrastructure side to models. [3:40] You're going to have folks from OpenAI and Anthropic [3:43] and a bunch of applications and agents. [3:46] So ask all your hardest questions. [3:48] Save them for the speakers. [3:50] They're going to love it. [3:52] And we're going to assign some readings. [3:55] So why should you take this course? [4:00] What should you achieve in this course? [4:02] What is a good thing to get out of this? [4:05] You know, honestly, I thought about this, [4:08] and I was just telling one of the students [4:09] here of how it all began. [4:11] I come back to campus once a year [4:14] and I talk about all that's happening. [4:17] And typically, it's in the context of finding great people. [4:21] I realized that this is such a big supercycle. [4:25] We know it. [4:26] We believe it. [4:27] At Altimeter, we have positioned our entire focus around it. [4:30] And I did not find a course that goes deep in a way [4:34] that I would have liked to be when I was an undergrad here [4:37] or a grad student here. [4:39] And I thought about in five years, [4:42] everybody was going to ask you, hey, did you see it coming? [4:45] You were at the start of it. [4:46] You were around when ChatGPT was launched. [4:48] You were around when the tectonic plates were forming [4:52] and clay was forming. [4:53] And I think you want to be able to say yes. [4:55] So half of you are going to start an AI company [4:57] and the other half are going to fund it. [4:59] [CHUCKLING] [5:00] So at least you should know where [5:02] to spend the Series A money that you're going to raise. [5:04] And at the very minimum, you'll have [5:07] a sense of what not to go, or at least have [5:10] mental models for, hey, this business that I'm [5:12] looking at or considering starting or considering funding [5:15] or considering joining, what are the right questions [5:17] to be asking? [5:18] What are the laws of physics that govern this business [5:22] at this part of the cycle? [5:25] I think it's going to be the biggest one yet. [5:27] And I'm excited that you guys are here to study alongside us. [5:37] I'm going to spend some time on this slide [5:39] because this is the punchline. [5:43] How many of you have seen a version of this before? [5:47] Well, those of you who did the readings, thank you. [5:49] I appreciate it. [5:51] We did include a notebook LM for those who [5:52] were more auditory inclined. [5:55] But let's talk about this for a second. [5:59] What is going on here is actually probably the biggest [6:04] question in generative AI right now, which is, [6:09] if you listen to any of the earnings calls [6:12] from the hyperscalers or even NVIDIA and others, [6:17] is we are investing so much into the CapEx. [6:23] We're investing so much into building these data centers. [6:26] It's a 5-layer cake, as Jensen calls it. [6:29] Energy, chips, power, interconnects, memory, [6:34] all that to give you a data center [6:36] that you can either rent by the hour or by the token, [6:39] that you can go train models on and serve those models. [6:43] And then the question is, hey, these models that you built, [6:46] are they creating economic value? [6:49] That is basically the right-hand side of this chart. [6:53] And to make an analog of the biggest technology [6:57] revolutions that I have seen, internet 25 years ago, [7:02] mobile 20 years ago, cloud, probably the most recent one, [7:05] 10 years ago. [7:06] I put up one of those charts on cloud, but on the readings, [7:10] you'll see the same for internet and mobile and cloud. [7:14] And that's the shape of the cloud ecosystem. [7:18] The cloud ecosystem looks dramatically [7:20] different than the AI ecosystem. [7:23] Anybody have guesses as to why that's [7:25] the case or your theory on why it's so different, [7:28] or reasons why this might look like-- [7:31] we're not going to call it a pyramid, [7:32] we're going to call it a triangle, inverted triangle. [7:38] Go ahead. [7:39] Is it because it's still early in the upside for AI? [7:42] Yeah, definitely. [7:43] [INAUDIBLE] [7:45] Definitely early. [7:46] That's a good guess, yeah. [7:48] Any others? [7:48] Any other thoughts? [7:51] Or maybe because NVIDIA has a monopoly, [7:53] so they can charge whatever. [7:56] Can you ask that question again next week [7:58] when we have the folks from NVIDIA? [8:01] But no, good, it's a great point. [8:03] They do have a stranglehold, right? [8:05] One of the charts we had in the readings [8:07] was the market share that NVIDIA has [8:08] on all of the compute right now, and it's up there. [8:16] Any other thoughts or hypotheses on why this is so different? [8:22] Yeah, I don't know, is it the cloud sectors seem [8:26] to be able to leverage the hardware to generate [INAUDIBLE] [8:29] and AI haven't really got there yet. [8:30] We know how software ate the world. [8:33] As Marc Andreessen said, software ate the world [8:35] because I could build software, you could build software, [8:39] and I could distribute it to millions of people. [8:41] And the marginal cost of running that software was close to 0. [8:45] These software businesses ran at 80% [8:47] some even at 90% gross margins. [8:50] That is not the case with this new economic model of AI, [8:54] because if we have a set of users using Cursor or using-- [9:01] you hear all these stories about large bit scale businesses that [9:04] are still not profitable at billions of dollars of revenue [9:08] scale, is because of that, is because the incremental user [9:12] of an AI application is not free. [9:15] It's not marginally free. [9:16] It's actually quite a bit more expensive to have AI users, [9:20] because it turns out you've got to burn those GPUs. [9:23] And I would say everything you guys [9:26] said from it being early to NVIDIA being dominant, [9:31] we'll call it, to the physics of the problem are very [9:36] different of how inference is run [9:38] is certainly where we are right now. [9:39] So I think that's the case right now. [9:41] I might add another dimension to it, [9:43] which I spoke about in the readings was, [9:45] we analyzed what happened in internet. [9:49] We analyzed what happened in mobile and cloud, [9:51] and how many years did it take for these triangles to flip? [9:57] And one of the examples we take is AWS. [10:03] AWS started in the year 2004. [10:06] AWS has its first customer in Netflix in 2010, [10:11] and ultimately Amazon shifted fully to AWS in 2012. [10:15] Eight years from breaking ground, [10:17] eight years from first CapEx investment cycle. [10:20] I don't know if any of you were around reading earnings reports [10:24] 20 years ago, but the big debate was, hey, [10:27] is Amazon going to go bankrupt? [10:29] And that was the biggest question everybody [10:33] had about the build out of AWS. [10:36] And thankfully, nobody, at least yet, [10:39] is on the verge of bankruptcy. [10:40] But these are large numbers. [10:44] So we'll come back to this slide. [10:46] But I would say this is the central theme of the course [10:49] that we're going to explore. [10:50] We're going to have speakers from some of the companies that [10:53] are listed here to others. [10:54] And the central theme that we're going to pick around [10:56] is like, hey, in your field, with the NVIDIA speakers, [11:00] are you a dominant force? [11:03] How long are you going to stay to be the dominant force? [11:05] What are the forces that you're most worried about? [11:07] Who are the ASICs that you're most worried about? [11:09] What are the pricing compression vectors for your business? [11:14] To the folks at Anthropic and OpenAI, [11:17] who we're going to talk a lot about profitability, [11:20] is you're serving a billion-user franchise at OpenAI. [11:26] With the Anthropic folks, honestly, 100% of this class [11:28] is on Claude. [11:29] So we'll ask them about, is this group of users profitable? [11:32] How do you think about profitability? [11:34] Is ads going to be a bigger source [11:36] of revenue than subscriptions? [11:37] And then for the folks in the middle, which [11:39] is the inference layer, this is the most competitive part [11:42] of the whole ecosystem. [11:44] There's a lot of startups that are doing really well. [11:49] They're winning so far. [11:51] But you've also got the hyperscalers [11:53] who want to have a dominant say in that layer. [12:00] So honestly, the jury's still out. [12:02] And the biggest question there is, are you [12:05] a feature or a platform? [12:07] A lot of new businesses that we are seeing on the infrastructure [12:12] side that they feel very good ideas. [12:15] But if you ask yourself the question, [12:17] hey, why is this not a part of AWS? [12:19] You are thinking about maybe it should be a part of AWS. [12:23] So for the speakers, we're going to talk a lot about that. [12:26] Any questions before we jump into the quiz? [12:30] Go ahead. [12:30] I'm curious how you think about-- so [12:32] on the right-hand side, like the triangle being-- [12:34] like the Application Layer being small, [12:36] how do you think about including incumbent platforms into that? [12:39] Maybe like Salesforce. [12:40] Maybe [INAUDIBLE] revenue. [12:42] Like, would you-- do you include them [12:44] as part of that-- in line with that pyramid [12:46] and how it shifts over time? [12:47] It's a great question and I might add, Salesforce, Palantir, [12:53] there's a series of-- let's call them "old economy businesses" [12:56] that are reinventing themselves to have skews of products that [13:01] are, in the case of Salesforce, Einstein, [13:05] in the case of Palantir, AIP. [13:06] In the case of-- there's a series of these. [13:09] And the answer is yes, there should be. [13:11] The answer is yes, that they should be. [13:12] The way I solve for that in this calculation [13:15] is I get the model revenue. [13:17] And so if you were running Salesforce, [13:18] you're probably running either one of the big models [13:22] or running inference. [13:23] So their spend is captured in the app layer [13:26] by way of the substrate. [13:28] It's very hard to extract that out from public disclosures. [13:31] But yeah, we should. [13:33] Yeah. [13:38] Is a large part of the bottom part of the right-hand side [13:41] pyramid, basically, buying capacity [13:43] for future revenue, which are the group at the top, which [13:46] is what we're not seeing in the [INAUDIBLE]. [13:49] It's a great question. [13:51] And maybe just to rephrase the question, the question is, hey, [13:54] is there a timing mismatch in the build out of the Semis [13:57] layer? [13:57] Because typically, you build Semis for a five-year period [14:01] or a six-year period. [14:03] But the application revenue is for right now. [14:08] It's a great question. [14:09] And that's what makes the lower half of this, [14:13] call it triangle, somewhat cyclical. [14:16] And you go through phases of CapEx cycles. [14:20] Think of it as like laying down the railroads. [14:23] That is very much the case. [14:24] And so there's a chart in the readings for what happened [14:28] in the mobile super cycle. [14:30] Something very similar happened. [14:31] The first inning had inflated market caps [14:34] for a lot of the CapEx heavy businesses. [14:38] And so if you think about a basket of CapEx names [14:43] to call it steady state names, you should expect that. [14:47] And so I suspect we're so early that that's happening as well. [14:53] I'm curious to see how you think about Google, because I [14:55] see that you labeled Google as Google Cloud [14:57] there on the [INAUDIBLE]. [14:58] But Google also have their own Gemini models [INAUDIBLE] TPUs, [15:02] and they're perfectly [INAUDIBLE]. [15:04] How many of you are kind of positioned in this triangle? [15:07] Any large conglomerate like Google deserves to be-- [15:12] we have to call business units. [15:14] So I would put the TPU business unit in Semis. [15:17] We include that here as we counted the revenue. [15:21] Their GCP unit is in the Infrastructure Layer. [15:24] And then the Gemini unit is at the Apps Layer. [15:30] We have a chart later that we'll talk a little bit about. [15:33] Gemini is actually one of the most used consumer applications. [15:37] It's the second most used consumer application right now. [15:40] And the biggest question there is, how much of that [15:43] is coming from the distribution advantage [15:45] that Google has to it meritocratic [15:48] being such a good application? [15:50] The jury's still out, but we'll get into it. [15:54] Yeah. [15:57] Well, let's-- go ahead. [15:59] I just have a question about prediction. [16:01] Right now, it looks like this triangle shape, [16:03] if it were to be successful, perhaps it [16:06] should become inverted. [16:08] But what does an unsuccessful new technology look like? [16:12] Does it stay a triangle? [16:13] How would you be able to predict whether this would be good [16:16] or not? [16:16] Yeah, I'm not sure the-- [16:21] maybe rephrasing your question, I might say it out like, [16:24] what is the stable equilibrium of this industry? [16:29] I think it is pretty clear that AI is unlikely to be a fad. [16:33] Is unlikely to be an unsuccessful endeavor. [16:37] And I think about the stable equilibrium of this chart [16:40] quite a bit. [16:40] In fact, I got into a little bit of a debate on Twitter [16:43] with somebody quite smart and who's [16:45] thought a lot about this exact question of what [16:48] is the stable equilibrium. [16:50] And my guess is that it might stay this way for longer [16:54] than I anticipated. [16:56] In the cloud, I think that range is about a decade. [17:01] I have a feeling this might stay longer-- [17:04] this way longer, because of just how hard it [17:07] is to get the substrate right. [17:11] But there will be one or two unlocks. [17:13] I couldn't tell you what they are, [17:15] but for example, if one of the ASIC programs at one [17:18] of the hyperscalers, be it Google's TPU or Meta's [17:22] MTIA or the folks at Amazon and OpenAI [17:26] and Microsoft and all the labs that we don't even know about [17:29] exist breakout success, I suspect [17:34] that'll be the biggest repricing of that layer. [17:39] The other catalyst could be I think [17:43] about the hyperscaler CapEx guidance in earnings calls, [17:46] which, by the way, I recommend everybody here to listen to. [17:47] Four times a year, you'll have public company CEOs tell you [17:50] their biggest questions, their biggest things that they're [17:53] thinking about, and I recommend listening to those. [17:57] If they just stopped guiding to big numbers [17:59] on CapEx, because that would imply [18:02] that the current equilibrium does not work. [18:05] So that's why the second thing that could happen. [18:07] And so you see, there's a lot of news about the guidance [18:10] that all the hyperscalers give about their CapEx [18:12] for that reason. [18:16] Go ahead. [18:16] [INAUDIBLE] of training versus inference. [18:20] Because my sense is if-- [18:22] the only way this flips is if inference is meaningfully larger [18:25] than training. [18:27] And I'm curious to hear your thoughts [18:29] on when you think that will happen, [18:31] because then you're seeing that [INAUDIBLE] [18:35] or you feel like they will stop spending [18:37] on training because they're not seeing [INAUDIBLE]? [18:39] [INAUDIBLE] [18:42] It's a great question, and it is probably [18:45] one of the nuggets of information [18:47] that NVIDIA's earnings calls, have the most sought [18:50] after nugget of what is NVIDIA's share of inference [18:53] in their fleet. [18:55] Last I checked, it was about 40%, [18:56] or they quoted to be about 40%. [18:58] Meaning that if they were selling a million GPUs, assuming [19:02] full utilization, about 40% of them [19:04] were used for inference and the other 60% for training. [19:07] I suspect that number will increase over [19:09] time in favor of inference, but I couldn't tell you [19:13] when and how it'll happen, because there's [19:15] a lot of training still going on in the world. [19:17] And the shape of the training workload, as you know, [19:19] looks very different from the shape of the inference workload. [19:22] A training workload is very predictable, high utilization [19:26] for a short period of time. [19:28] The inference workload is very burst usage. [19:32] Typically when humans are awake until the agents take over, [19:36] maybe then it'll be 24/7, and harder to predict. [19:41] It goes down around Christmas for some reason. [19:43] It goes down around Thanksgiving for some reason. [19:45] But I think that might be the case, though we, [19:47] at least in this calculation, we try to capture it [19:50] because it's a mix and-- [19:53] but it's a good hypothesis. [19:57] [INAUDIBLE] where would probability be? [20:01] It's on slide 16. [20:02] We'll come to it. [20:03] We'll come to it. [20:04] I'll give you the answer. [20:05] The most profitable part of the stack [20:07] is the Semis layer by a long shot. [20:10] NVIDIA's data center revenues or on a gross margin of-- [20:13] About 75%. [20:14] Don't quote me on it. [20:15] It's like plus or minus a couple percentage points from there. [20:19] Whereas, I estimate some of the Application Layer revenues [20:23] to be somewhere between-- depending on who you ask, [20:25] between 0% and 30%. [20:27] And so the gap is quite wide. [20:31] And I think the reason to that-- [20:33] I mean it's a theory that a gentleman here had is, [20:37] there's one player who runs the tables on the Semis here. [20:42] And so it's very much the case. [20:44] And in fact, if you looked at this [20:45] from a profitability perspective, [20:47] it's even more concentrated. [20:49] The triangle is even more concentrated. [20:53] I'll flash that in a second when we get to it [20:56] but it's a great question. [20:57] Go ahead. [20:59] Do you think [INAUDIBLE]? [21:07] Yeah, that is definitely a big part of it, [21:09] is that we've gone through the investment cycle in cloud. [21:13] It's definitely an element to it. [21:16] Go ahead. [21:17] If all these Infra companies like Google, AWS [INAUDIBLE] [21:23] all their own TPUs, and media is also doing inference, [21:27] OpenAI is also searching for some ASICs, [21:30] where does all these ASICs inference [21:33] startups want to sell to if they are using their own chip? [21:39] There's $300 billion of revenue to fight about. [21:45] But to answer your question, about half [21:46] of that, as Jensen discloses on the earnings calls, [21:49] is from the big hyperscalers. [21:52] So those are probably going to be your primary customers. [21:55] So if you were starting a chip company today, [21:58] you would have a very-- the shape of your customer base [22:00] is a very small number of very large orders. [22:06] It's a very different shape from building [22:07] a consumer business or an enterprise software business. [22:10] And then you might have a long tail of other enterprises, [22:13] though I wouldn't bank on it because I think they just [22:15] go to the cloud providers. [22:19] If you were thinking about starting [22:20] a chip company, it should be your number one consideration, [22:23] is like, which of the five are you going to sell to first? [22:30] Last question. [22:31] [INAUDIBLE] do you get a small handful of winners [22:34] in each of these layers? [22:35] It takes multiple years for that to play out. [22:39] Maybe I'm wrong, but I feel like in the past [22:41] there hasn't been a fully vertically integrated layer [22:44] [INAUDIBLE]. [22:46] I get that Google is fully vertically integrated [22:48] on the right-hand side, but wondering [22:49] how that shift the balance of power this side. [22:51] What a great question. [22:53] The biggest winner on the internet super cycle [22:55] is probably Google. [22:57] It's about $3 trillion in market cap, has near 99% market [23:00] share in search. [23:02] I would say that that's a pretty vertically integrated player, [23:05] right? [23:05] They run their own file server to search to ads on top [23:09] to the user experience. [23:11] Let's see, the next one is Mobile. [23:13] The winner of that Super cycle is Apple. [23:14] What? [23:15] With $2.5 trillion or so in market cap. [23:18] You called it already. [23:19] The next one is, let's say social. [23:22] Meta is probably the big winner in social. [23:26] They're not as fully integrated. [23:28] And what is their market cap like? $2 trillion [23:31] or something right now? [23:32] Pretty dominant, but maybe they lost a trillion [23:34] because they didn't fully go down to the servers. [23:36] And then the cloud is fairly heterogeneous. [23:39] We don't have a single player that won the cloud. [23:41] You've got the three big oligopolies [23:43] in AWS, GCP and Azure, but they're not fully integrated. [23:46] And NVIDIA has been trying a lot. [23:49] NVIDIA has been trying-- [23:50] I don't know if you've heard of DGX Cloud, which [23:52] is their cloud effort to build the cloud ecosystem. [23:55] Obviously, they've got a series of vertical apps [24:00] that they're trying. [24:00] So yeah, you might be on to something. [24:07] Folks, I know it's a Thursday evening at 5 o'clock, [24:09] probably the last thing standing between the weekend [24:12] and your weekend. [24:13] So I don't want to be that person. [24:15] So I'm going to jump into the part that wakes you up. [24:17] I do actually have-- this is a quiz [24:19] that we're going to go through. [24:20] I'm going to give you a hint about the companies [24:24] that we're going to go through. [24:25] I do have a prize for the winner. [24:28] This is the prize so you're motivated. [24:31] And you win points on two grounds. [24:35] One is by being right, and the other is by being fast. [24:40] The fastest way to be fast is to do fast inference [24:44] and drop that thing into Claude. [24:46] Please, you're welcome to do that. [24:48] Just give the human players five seconds, let them go at it [24:53] and let them win the analog way. [24:55] And if you really want to use Claude, [24:57] you're welcome to do it, just give them five seconds. [25:00] All right, so this is question number one. [25:09] [CHATTER] [25:25] Ready for the next? [25:32] The software engineers in the room [25:34] might have an unfair advantage. [25:51] So I wanted to spend the next maybe 10 minutes [25:55] or so going into some of the hypotheses [26:04] that I have about what's going on [26:06] and why the value is accruing in the manner that it is. [26:09] I think there was a question, a very good question, [26:11] about profitability and how it gets magnified. [26:14] So we'll jump through that. [26:16] But again, feel free to stop me if you have any questions. [26:19] I have a feeling we're going to have very little time left, [26:21] and I do want to end on time. [26:24] You guys remember this? [26:27] And I painted it slightly differently [26:29] on the next chart, which is, I did the same exercise that I [26:32] did that I posted about two years ago. [26:37] And what it looked like two years ago [26:39] was this thing on the left, where the ecosystem was [26:44] obviously a lot smaller. [26:45] It was about five times smaller. [26:47] Shockingly, the shape of it hasn't changed much. [26:52] This is despite heroic growth. [26:55] And if you look at the revenue that was added, [26:58] about $350 billion or so of revenue [27:01] added, a good, like, 75% of it just went straight to Semis [27:07] in the last two years. [27:08] Despite apps having grown more than 10x, [27:16] it still hasn't made that big of a dent. [27:18] And so I was like, OK, well, let's dig deeper into this. [27:21] If you started to open up each of these cells [27:23] and you're like, hey, what companies make up [27:25] each of those parts? [27:28] Most of that 300 is in NVIDIA, as you guys know. [27:32] the Apps is actually, two companies make up about 90% [27:36] of it. [27:36] Anybody want to guess which those two are? [27:39] The Infra segment is the one that has the most competitive [27:43] intensity, as we discussed. [27:44] It is probably the place where there's [27:46] the biggest battle brewing both sideways, but also [27:49] across the stack. [27:51] It's also the place that has the highest metabolic rate in that [27:54] there's a lot of companies being formed, [27:56] there's a lot of companies that are getting bought out. [27:58] And I would say it's the most competitive, [28:02] but also the most unstable of the equilibriums [28:04] that we have right now. [28:06] And the question that we think about [28:08] as we think about investing, as you guys will think [28:10] about investing your time, is how much time will this [28:13] chart that has moved such little in the last two years, what [28:18] is the amount of time it will take to get to cloud software [28:21] like shape? [28:24] Is it five years? [28:25] Is it 10 years? [28:25] Is it 15 years? [28:27] Is it never? [28:28] Maybe it just stays that way. [28:30] We do think it will happen. [28:31] We think it will happen at some point, [28:32] but it's not happening nearly fast enough. [28:37] The second thing that we've been thinking a lot about [28:39] as we think about the future of AI is-- [28:44] I don't know if you guys saw this chart in the readings, [28:46] but consumer AI, which is the biggest, call [28:49] it market for AI right now outside of coding, [28:53] has incredibly high usage on-- [28:57] ChatGPT, most of it is free. [28:58] About 95% of the users are free. [29:02] And Gemini, whose-- I don't know if you guys saw, but Demis, [29:05] who leads DeepMind, announced that they were not [29:08] planning to do ads as a subscription-- as a revenue [29:11] model. [29:12] We've been thinking a lot about hey, how [29:14] big do these businesses get? [29:16] What is the monetization engine of these businesses? [29:18] Do you think a subscription business will be larger [29:21] or ads business will be larger? [29:23] And so what I did was I looked at the largest consumer [29:28] franchises outside of AI. [29:32] And so you will see that-- [29:33] I mean, you all know these products. [29:36] There's a class of products that have gotten [29:38] to three billion users scale. [29:40] These are almost near mandatory products to live your lives. [29:44] This is WhatsApp and Chrome, which [29:46] you could not live without. [29:48] Then there's a class of products that the 1,5 billion to 2 [29:51] billion users scale, which are social, [29:52] these are social products like Instagram, TikTok, and Facebook, [29:56] they're not mandatory, but they're exhibit very good [29:59] network effects. [30:00] If my friend Chloe is on one of these, [30:02] I'm more likely to be there. [30:04] And then you've got the third category of mainstream consumer [30:08] products that are neither mandatory, that are neither [30:12] extremely social, but I would call niche products. [30:15] If you're shopping, you're going to Amazon. [30:17] If you're looking for music, you're going to Spotify. [30:19] If you're looking for a good debate, [30:20] you're going on Twitter or cat videos. [30:23] Any guesses on where closer to which of these will [30:26] ChatGPT and Gemini are right now? [30:29] And the answer is on the next slide, so we'll get it quickly. [30:33] Would you guess that ChatGPT or the leading AI application's [30:36] terminal scale will be closer to a mandatory app like YouTube [30:41] or WhatsApp, a social app like Instagram or TikTok, [30:46] or a niche app like Spotify or Twitter? [30:52] Any guesses. [30:52] If not, I'll reveal the question. [30:54] Go ahead. [30:55] I would say on the YouTube, WhatsApp scale [30:57] because it would be a daily utility. [30:59] Yeah. [31:00] People would just be using it daily [31:01] as part of their normal life. [31:02] Yeah. [31:04] You are-- well, let me show you the answer [31:08] and we'll come back to your biases. [31:10] Any other guesses? [31:10] Any different guesses? [31:11] Go ahead. [31:12] I think it's closer to Facebook time. [31:15] [INAUDIBLE] [31:19] Yeah. [31:20] Yeah, that's right. [31:21] You're certainly right right now. [31:24] OK, I'll show you guys the answer. [31:28] This is how they fare if you plot them all together. [31:31] ChatGPT has just overtaken the niche app category. [31:35] Gemini still has not. [31:36] You were right that it's heading towards social. [31:39] Personally, I would have loved, as an investor at OpenAI, [31:42] I would have loved for it to start heading [31:44] towards the core utility. [31:45] But one of the biggest questions that we [31:47] ask ourselves is, is knowledge work work that everybody does? [31:52] Is the work of-- [31:54] ChatGPT is not a place where you're messaging other folks, [31:57] yet it's not a place where you're getting your email inbox [32:00] or your dopamine fix. [32:03] It's a place where you go and you have to do active work. [32:05] You have to go ask a question. [32:08] And the number of people in the world who [32:09] are asking active questions of technology [32:12] is not the entirety of the population that's online. [32:16] There's about 8 billion people on the planet. [32:18] 4 billion of them are online. [32:20] The rough economics of consumer applications, Alphabet [32:25] has about 4 billion users. [32:27] They monetize them at about $100 a user a year. [32:30] Meta's got about 3.5 billion users [32:33] that monetize at about $70 a user a year. [32:36] The leading AI provider ChatGPT, has [32:41] got about a billion users that are monetized [32:43] at about $10 a user a year. [32:45] And so the question is, how do we get the billion up to 4 [32:49] billion? [32:50] I'm not sure knowledge work is the answer. [32:52] I think we'd have to go beyond knowledge work. [32:55] And then the second question is, how do we [32:56] get the $10 a user per year up from 10 to 100? [33:02] And I'm not sure subscription is the answer. [33:04] I suspect we'll have to go into ads. [33:06] And I suspect the ads that ChatGPT will be able to serve [33:09] or Claude will be able to serve will have a lot better pricing [33:11] because they will understand your intent, [33:14] that you will be logged in, very good attribution, a lot more [33:16] trust. [33:18] And I think that will be the other big headline this year. [33:21] And you heard it here first. [33:23] It'll be a big deal. [33:24] There's a lot of alpha in understanding the ad [33:28] model really well. [33:29] Once again, 10 years ago, the Facebook IPO, there [33:32] was a lot of short reports on Facebook [33:33] because people said, hey, well, these ads worked on a computer, [33:37] they're not going to work on a phone. [33:39] Why? [33:39] Because there's no space on a phone. [33:42] Shocker, we found the space on a phone. [33:46] The same thing's going on right now, [33:47] which is, while I'm having this conversation, [33:50] it is a very personal conversation, [33:52] I don't want to be interrupted by advertisements. [33:54] That's the bigger debate. [33:57] I couldn't tell you what it's going to be like, [34:00] but I am optimistic that we will find it. [34:03] And I think that's going to be a big unlock, a big unlock [34:06] for this economic model. [34:08] And so we'll dig into that in one of the speaker sessions [34:11] later this year. [34:14] I've got a bunch more slides. [34:16] We are at time. [34:17] Thank you.