NVIDIA Introduces NVLink Fusion and Custom NVHBM Base-Die Technology for AI Accelerators

NVIDIA Introduces NVLink Fusion and Custom NVHBM Base-Die Technology for AI Accelerators NVIDIA has announced NVLink Fusion and NVHBM, an interconnected platform and custom memory base-die technology designed to integrate custom hyperscaler accelerators (XPUs) into NVIDIA's rack-scale AI infrastructure. NVHBM provides up to 30% more memory bandwidth per stack and up to 15% lower memory power consumption compared to standard HBM4e. A redesigned custom PHY and moving the memory controller into the 3D HBM stack cuts interface area by up to 67%, yielding up to 25% to 30% more compute die area. NVLink Fusion uses a chiplet to connect custom XPUs across racks via NVLink and to CPUs via NVLink-C2C, delivering up to a 30% overall end-to-end performance gain per XPU. In a 1-gigawatt data center running 2,000W XPUs, the memory power savings can free up capacity for up to 15,000 additional XPUs. Bandwidth and Memory Interface Redesign NVHBM delivers up to 30% more memory bandwidth per stack compared with standard HBM4e, addressing data starvation during bandwidth-intensive AI training and inference. To overcome package area constraints, NVHBM integrates a custom physical interface (PHY) and moves the memory controller directly into the 3D HBM stack. Compared to the JEDEC HBM4e standard, this shrinks the PHY and support footprint by up to 67% and simplifies interposer routing, allowing up to 80% more usable silicon across the layout and up to a 30% expansion in central compute die space. Power Efficiency at Data Center Scale The custom base die reduces HBM power usage by up to 15% relative to standard HBM4e. This reduction lowers thermal strain on accelerator packages and data center cooling while freeing power headroom for compute logic. In large facilities, such as a 1-gigawatt data center utilizing 2,000W XPUs, compounding these memory efficiency gains creates enough power headroom to deploy up to 15,000 additional accelerators. Rack-Scale Integration with NVLink Fusion NVLink Fusion links custom XPUs into NVIDIA's broader ecosystem and MGX rack-scale architecture via an NVLink Fusion chiplet. This connects rack-level XPUs into unified scale-up compute domains supporting workloads like expert parallelism, while upstream connections to CPUs are handled via NVLink-C2C. Combined with NVHBM's bandwidth, area, and power improvements, the unified architecture yields an estimated 30% overall end-to-end performance improvement per XPU.

NVIDIA 推出 NVLink Fusion 与 NVHBM：赋能定制 AI 加速器与机架级基础设施 NVIDIA 推出 NVLink Fusion 技术与定制 NVHBM 基础芯片方案，允许超大规模企业将定制 XPU 及 CPU 接入 NVIDIA AI 基础设施平台，从带宽、面积和功耗三个维度优化加速器设计，并实现机架级扩展。 NVHBM 相比标准 HBM4e 可提供最高提升 30% 的内存带宽、降低 15% 的 HBM 功耗，并释放多达 25% 的计算芯片面积。 通过将内存控制器移入 3D 堆栈并集成定制 PHY，NVHBM 相比 JEDEC HBM4e 减少了高达 67% 的物理接口及支持区域。 NVHBM 带来的能耗节约在 1 吉瓦的数据中心规模下，可为 2000W 功耗的 XPU 释放容纳多达 15,000 个额外加速器的电力裕量。 NVLink Fusion 通过专有小芯片将定制 XPU 桥接至第六代 NVLink 互联网络，与 NVLink-C2C 配合构建统一的机架级扩展域。 NVHBM 破局内存带宽与芯片面积限制 针对 AI 训练与推理中日益严重的数据供给瓶颈，NVHBM 单堆栈内存带宽较标准 HBM4e 提升达 30%，显著加快了 KV 缓存及模型权重的读取。此外，NVHBM 重构了物理内存接口，将内存控制器移至 3D 堆栈内部，使 PHY 及辅助区域缩减达 67%，简化了中介层布线，并为中央计算芯片释放了多达 30% 的额外面积，便于设计人员集成更多特定负载计算单元。 功耗优化助力数据中心大规模扩展 在电力受限的现代 AI 设施中，NVHBM 相比 HBM4e 可减少 15% 的内存功耗。这一节能效果不仅优化了单卡能效与散热设计，在大规模部署下更具杠杆效应：在一座采用 2000W XPU 的 1 吉瓦级 AI 工厂中，节省的电力能够支持额外扩展多达 15,000 个 XPU，对降低大模型长推理场景下的能耗成本尤为关键。 NVLink Fusion 构建机架级统一互联 NVHBM 解决芯片级性能问题的同时，NVLink Fusion 通过连接小芯片（chiplet）将定制 XPU 接入第六代 NVLink 扩展网络，并将 XPU 通过 NVLink-C2C 与 CPU 互联，将整机架加速器融合为一个 scale-up 域。结合带宽、面积与功耗的协同优化，每个 XPU 可获得约 30% 的端到端综合性能提升，同时允许定制 XPU 与 GPU 混合异构运算，加速定制 AI 工厂的落地与复用。

AI factories must support increasingly large models and more complex reasoning workloads . To keep up with the insatiable compute demands of AI workloads, hyperscalers and AI-native companies are developing custom AI accelerators, or XPUs. Deploying these accelerators at scale requires high-bandwidth memory (HBM) to keep compute fed, sufficient package and silicon area for more compute, efficient power delivery, and a resilient supply chain. It also requires a rack-scale architecture for deploying  XPUs into data center infrastructure. NVIDIA NVLink Fusion is the connective technology and IP that enables hyperscalers and AI natives to deploy custom XPUs and CPUs into the NVIDIA AI infrastructure platform. They can use the NVIDIA scale-up and scale-out technology stack, ecosystem, and MGX rack-scale architecture to reduce development and deployment complexity, improve performance, and accelerate time to market for semi-custom AI factories. At the package level, NVHBM, complements this unified architecture. NVHBM is a custom HBM base-die technology designed and validated with leading memory vendors that enables increased memory bandwidth, better area savings, and lower power consumption. These improvements can help custom XPUs support larger models, read KV cache data faster, and improve training and large-scale inference. Why bandwidth, die area, and power drive accelerator design Training, inference, and agentic AI workloads increasingly depend on high-throughput access to model weights, KV cache, and activation data. As AI systems scale from individual accelerators to rack-level compute domains, the accelerator package must balance compute logic, power delivery, thermal design, and high-bandwidth memory. HBM places vital memory bandwidth close to the accelerator, but qualifying leading memory technology, package integration, and validation can become a bottleneck for custom accelerator programs. Through NVLink Fusion, customers gain access to NVHBM base dies that are validated with leading memory manufacturers, helping reduce integration and qualification bottlenecks. Feature NVHBM benefit Bandwidth Up to 30% more memory bandwidth compared with standard HBM4e Area More efficient interface connections allow up to 25% more compute die area for additional XPU capabilities Power Up to 15% lower HBM power usage compared with standard HBM4e adds up savings across thousands of XPUs Table 1. NVHBM brings three main platform-level advantages to AI accelerator programs: higher memory bandwidth, more package and silicon area, and lower HBM power usage The memory bandwidth bottleneck in modern AI accelerators AI accelerator performance depends on how consistently compute engines are supplied with data. Higher HBM speeds increase usable memory bandwidth within a given package budget, improving the ability to serve bandwidth-intensive phases of training and inference. NVHBM delivers up to 30% more memory bandwidth per stack compared with standard HBM4e. For memory-bound or partially memory-bound AI workloads, that translates into better accelerator utilization and higher throughput. This can increase per-user token throughput during large-model inference by moving data between HBM and compute cores faster, keeping them fed. While NVHBM increases memory bandwidth within each accelerator, NVLink Fusion connects accelerators across larger domains so workloads can use distributed compute and memory more efficiently. This scale-up domain is especially critical when using advanced routing techniques like expert parallelism (EP) or WideEP . In these scenarios, different experts reside on different GPUs and require seamless, high-speed synchronization across the entire rack. NVIDIA NVLink , the scale-up networking fabric for AI factories, transfers activations and hidden states between experts and helps synchronize distributed caches across the scale-up fabric. NVHBM minimizes data starvation by keeping the local compute engines consistently fed. More package area, more flexibility For custom AI silicon, every square millimeter matters. Accelerator designers must decide how much area to allocate to matrix engines, vector units, on-chip SRAM, cache hierarchy, control logic, memory interfaces, network-on-chip, and scale-up connectivity. A custom memory implementation can help reduce the design and package overhead associated with accessing HBM, freeing up area for workload-specific capabilities. As AI workloads diversify, this additional die area gives hyperscalers more flexibility to optimize XPUs for inference serving, recommendation systems, multimodal pipelines, or internal training workloads. By reducing the area required for the memory interface, NVHBM enables teams to dedicate more of the chip directly to performance. Area savings are achieved primarily through a redesigned physical memory interface (PHY). Standard HBM relies on wider interface connections, increasing the total package footprint. NVHBM uses a custom base die optimized for efficiency, featuring reduced I/O area requirements achieved by moving the memory controller into the 3D HBM stack and integrating a custom PHY. Compared with the JEDEC HBM4e standard, this design reduces PHY and support area by up to 67%. The narrower interface also simplifies interposer routing, providing up to 80% more usable silicon across the entire layout. Figure 1. Comparison of die area savings with NVHBM compared to standard HBM As shown in Figure 1, shrinking the memory interface connections enables the central AI compute die to expand into the newly freed space. This reclamation provides up to a 30% increase in available main-die silicon for compute or other features. The additional silicon area enables XPU designers to add more capabilities within a fixed package footprint. Power savings for efficient scaling Power is one of the hardest constraints in modern AI infrastructure. HBM power contributes to the accelerator power budget, package thermal design, rack power envelope, and data center cooling plan. NVHBM enables 15% lower HBM power usage compared to standard HBM4e, creating additional power and thermal headroom for compute. Power savings matter at multiple levels. At the XPU level, lower HBM power can improve performance per watt and create room for more compute or higher sustained utilization. At the rack level, it can help reduce pressure on power delivery and cooling systems. At AI factory scale, even modest reductions in memory subsystem power can add up across thousands of accelerators. When compounded across an entire 1-gigawatt data center using 2,000W XPUs, power savings can enable up to 15,000 additional XPUs in compute headroom. The benefit is especially important for large-model inference. XPUs must repeatedly read model weights and KV-cache data while serving users at low latency and high throughput. Reducing the energy spent moving that data can help support faster inference on large models, larger batch sizes, and more efficient use of deployed power. Combining NVLink Fusion with NVHBM at rack scale NVHBM boosts XPU performance and efficiency at the chip level. NVLink Fusion enables hyperscalers and AI natives to connect their XPUs to the rest of the NVIDIA AI platform. By compounding a 30% increase in memory bandwidth, 25% more die area, and 15% HBM power savings, these co-designed architectural improvements translate into a significant 30% overall end-to-end performance increase per XPU. This connectivity is achieved through the NVLink Fusion chiplet, which bridges custom XPUs and the NVLink fabric, connecting all XPUs in a rack into a single scale-up domain. Now in its sixth generation, NVLink is the only proven, purpose-built scale-up networking fabric for AI factories, delivering leading performance and intelligent resiliency. Upstream, the XPUs can connect to the CPUs via NVLink-C2C. NVLink Fusion adopters can combine custom XPUs and CPUs with NVIDIA scale-up and scale-out technology stack and ecosystem to reduce development and deployment complexity, increase performance, and accelerate time to market for semi-custom AI factories. And by standardizing on a single unified architecture, NVLink Fusion simplifies operations across the data center, enables flexible reprovisioning of data center capacity, and enables custom AI XPUs to integrate with GPUs for heterogeneous compute. The next phase of custom AI silicon NVLink Fusion provides a common scale-up foundation for GPUs, custom XPUs and CPUs, networking, and rack-level software. NVHBM complements that foundation with greater memory performance, compute density, HBM power efficiency, and supply resiliency for next-generation accelerators. These technologies give partners a more direct path from custom AI accelerator design to rack-scale deployment and production volume. Learn more about NVLink Fusion and industry adoption of NVHBM .