NVIDIA Integrates Groq 3 LPX Deterministic Execution into Vera Rubin Platform for Power-Efficient AI Inference

NVIDIA Integrates Groq 3 LPX Deterministic Execution into Vera Rubin Platform for Power-Efficient AI Inference NVIDIA is incorporating the Groq 3 LPX low-latency accelerator into its Vera Rubin platform to improve performance per watt during high-interactivity inference. By relying on a cycle-exact deterministic execution model across 256 LPU chips per rack, the system predicts current demands down to the clock cycle, enabling proactive power and clock management. Factory-level power software (DSX MaxLPS) and rack-level smoothing permit up to 40% more GPUs within site envelopes and 35% higher token throughput. Groq 3 LPX leverages cycle-exact deterministic scheduling across 256 LPUs to predict current draw, removing dynamic execution unpredictability. Preemptive Power (PEP) and Clock Period Synthesis (CPS) proactively adjust voltage and clock cycle lengths, lowering di/dt rates and reducing voltage droop by over 60%. Shrinking the required voltage guardband decreases power consumption by a low-double-digit percentage, with Vera Rubin NVL72 and Groq 3 LPX achieving up to 35x higher throughput per megawatt for 2T+ parameter models compared to GB200 NVL72. Groq 3 LPX power controls are scheduled to arrive on the Vera Rubin platform in the second half of 2026. Power Challenges in AI Factories Power availability serves as the primary ceiling for AI compute, turning performance per watt into the critical benchmark over raw throughput. NVIDIA addresses this through the Vera Rubin NVL72 platform using DSX MaxLPS software to shift power across racks, alongside rack-level capacitors and Intelligent Power Smoothing software to absorb sharp transient spikes. For highly interactive tiers with strict latency demands, the architecture introduces the Groq 3 LPX accelerator rack to maximize output within tight power constraints. Voltage Droop and Guardband Inefficiencies Conventional AI chips schedule compute dynamically, causing rapid transistor switching during intensive matrix multiplications. These sudden transitions draw immediate current from local decoupling capacitors faster than voltage regulators can compensate, causing voltage droops governed by the rate of current change (di/dt). To avoid dropping below minimum operational voltage (Vmin) and generating compute errors, chips traditionally maintain an excess voltage guardband. Because power scales with the square of voltage, running elevated guardbands consumes substantial energy. Cycle-Exact Predictability via Deterministic Architecture Groq 3 LPX eliminates dynamic scheduling uncertainty through an architecture featuring specialized units—MXM for matrix operations, VXM for vectors, and SXM for reshapes—alongside hierarchy-free on-chip SRAM banks and direct interchip links. Every compute step, memory transfer, and interchip communication requires an identical number of clock cycles across runs. The LPU compiler uses this determinism to map all operations and resolve hardware resource conflicts before execution, accurately plotting current draw cycle by cycle across the entire 256-chip rack. Proactive Voltage Shaping via PEP and CPS Using this predictable current curve, the compiler orchestrates Preemptive Power (PEP) and Clock Period Synthesis (CPS). PEP signals the power delivery network to begin adjusting supply voltage ahead of sudden workload demand surges, reducing the burden on decoupling capacitors. CPS leverages a plesiosynchronous clock system to dynamically lengthen clock cycles during severe current spikes, dampening the di/dt rate. In internal benchmarks, combining PEP and CPS reduced voltage drop by over 60%, cutting baseline voltage demand by high single-digit percentages. Platform Impact and Deployment By minimizing the voltage guardband, Groq 3 LPX reduces the energy needed for equivalent inference workloads by an estimated low-double-digit percentage. Scheduled to debut on the Vera Rubin platform in H2 2026, the technology works with factory-level management to deliver up to 35x higher throughput per megawatt compared to the previous-generation NVIDIA GB200 NVL72 when serving 2T+ parameter models at long context lengths.

NVIDIA解析Groq 3 LPX确定性执行技术：以精确时钟调度降低功耗并提升Vera Rubin平台推理能效 在AI工厂受限于供电预算的背景下，NVIDIA阐述了Groq 3 LPX低延迟加速机架如何通过确定性执行模型，预测并平抑电流波动，大幅缩减电压保护裕量，从而提升Vera Rubin平台在高交互AI推理场景中的每瓦性能。 AI芯片因突发算力需求导致电流骤增，进而产生电压降（voltage droop），传统方案需维持过量电压保护裕量（voltage guardband）以避免低于最低运行电压（Vmin）。 Groq 3 LPX通过包含MXM、VXM、SXM和片上SRAM的架构实现全机架256颗LPU的确定性执行，编译器可提前生成精确到时钟周期的计算、数据流及电流需求曲线。 基于电流预测，抢占式供电（PEP）提前指示供电网络调整电压，时钟周期合成（CPS）动态调整高负载周期的长短以降低电流变化率（di/dt）。 内部测试显示该方案可减少超过60%的电压降，使基准电压降低高个位数百分配比，因功耗与电压平方成正比，从而显著削减运行功耗。 Groq 3 LPX预计于2026年下半年引入Vera Rubin平台；针对2T+参数大模型，该组合相比GB200 NVL72在长上下文与高交互场景下每兆瓦吞吐量最高可提升35倍。 AI供电瓶颈与电压裕量挑战 在矩阵乘法等密集计算中，晶体管快速开关会导致芯片电流在纳秒级内剧增。尽管去耦电容与电压调节器能提供电荷，但电路电感会阻碍电流瞬时上升，引发临时性电压降。若电压跌破芯片正常运作的最低阈值（Vmin），将导致计算错误。为了规避瞬态最坏情况，常规芯片必须持续维持较高的电压保护裕量。由于功耗与电压的平方成正比，额外提供10%的电压将带来21%的持续功耗浪费。 Groq 3 LPX周期精确调度与电流预测 Groq 3 LPX机架内256颗LPU采用确定性执行模型。其单一加速器包含专门处理矩阵乘法的MXM、矢量计算的VXM、转置变形的SXM，以及无层级的片上SRAM，且芯片间采用直连通信。在此架构下，计算、访存与通信在不同运行中消耗的时钟周期完全恒定。编译器可以在负载执行前规划出周期级精确的任务与数据传输路线，提前化解存储冲突，并据此提前精确推导出整机在每一个时钟周期的电流需求曲线。 主动调控技术：PEP与CPS协同降耗 掌握时钟级电流曲线后，Groq 3 LPX采用了两项互补的主动调控技术。抢占式供电（PEP）让芯片在电流峰值到来前提前通知供电网络调节输送电压，减少去耦电容的放电压力；时钟周期合成（CPS）则依托准同步时钟系统，在尖峰时刻微调单个时钟周期的长度，从而拉缓电流变化率（di/dt）。内部测试表明，这套机制使瞬态电压降减少超过60%，使系统能够安全运行在更低的基准电压下，实现低双位数百分比的整体功耗削减。 整合至Vera Rubin平台与整体能效飞跃 Groq 3 LPX计划于2026年下半年引入NVIDIA Vera Rubin平台，与工厂级的DSX MaxLPS调度软件及Vera Rubin NVL72机架级电容平滑技术相配合，形成全方位的功耗优化体系。在面向2T以上参数模型的高交互及长上下文推理任务中，将Groq 3 LPX与Vera Rubin NVL72结合使用，相较上一代NVIDIA GB200 NVL72，每兆瓦电力的Token吞吐量最高可提升达35倍。

Power is a defining constraint for AI factories . As AI workloads demand a full compute platform to serve them, each component of that platform must maximize output within the factory’s limited power budget. This makes performance per watt—rather than raw, unnormalized throughput—the ultimate measure of an AI platform’s value. The NVIDIA Vera Rubin platform is designed to enable power-efficient AI at scale. At its core is NVIDIA Vera Rubin NVL72 , which delivers strong performance per watt across the widest range of AI compute demands—from throughput-optimized large batches to the small batches of high-interactivity tiers, for both open and closed models. Factory and rack-level power management innovations drive Vera Rubin performance on this important metric. At the factory level, NVIDIA DSX MaxLPS software shifts power between racks as workloads demand shifts, allowing an operator to recover stranded power and provision up to 40% more GPUs within the same site-power envelope and deliver 35% higher token throughput . Inside each Vera Rubin NVL72, rack-level capacitors, along with state-of-charge Intelligent Power Smoothing software, absorb the bursty power spikes of training and inference workloads, so the factory can be planned around sustained demand instead of worst-case peaks. This means more deployable compute per megawatt. The highest interactivity tiers present unique challenges. For these, the platform adds NVIDIA Groq 3 LPX as a low-latency accelerator. This post explains innovations within individual LPX racks that make it a power-efficient contributor to the Vera Rubin platform, including its deterministic execution model. What are the power-efficiency technologies in the NVIDIA Groq 3 LPX rack? The Groq 3 LPX deterministic execution model allows the LPU compiler to create a schedule of exactly when each piece of data will move to which specific compute unit and precisely when that operation will execute, down to the clock cycle. It extends to all 256 LPU chips in the rack, enabling ultrafast interactivity at long context and power management techniques that take advantage of this determinism. After the LPU compiler has generated the workload execution schedule, it can predict the electrical current draw for each cycle in that schedule. This in turn enables two complementary technologies: Preemptive Power (PEP): Prepares the power-delivery system for changes in demand before they arrive Clock Period Synthesis (CPS): Shapes how abruptly demand rises and falls Together, PEP and CPS help reduce the “voltage guardband” or “electrical safety margin” that must be continually provided to any set of chips, despite not directly powering the AI workload. Decreasing this voltage guardband means a greater proportion of scarce power can be spent on the workload. Figure 1. Groq 3 LPX determinism enables technologies like PEP and CPS to facilitate more energy efficiency for the same workload What are the challenges involved in delivering power to AI workloads exactly when needed? To run AI workloads, chips execute a series of instructions ranging from simple data reshapes to complex, power-intensive matrix multiplications. Nearly all chips include hardware features that enable dynamic switching between these myriad operations while the workloads are running, as resources become available. This flexibility means that a compute-intensive operation may be scheduled to be executed at any moment. The system that delivers current to the chips must be designed to handle this. The matrix multiplies and vector multiplies that power AI workloads involve a high amount of transistor switching in a short period of time, which increases the chip’s current demand on a nanosecond scale. The chip draws that extra current from the closest available source: capacitors that sit next to the chip known as decoupling capacitors (or decaps). Discharging these decreases the chip’s voltage. The chip’s board does have a voltage regulator—a dedicated circuit whose job it is to hold the chip supply voltage at a set target—but it cannot respond instantaneously due to inductance , which resists any rapid rise in current it delivers. Nevertheless, the current the voltage regulator supplies eventually catches up with the chip’s demand, allowing the voltage to recover. These temporary voltage drops, ultimately driven by sharp changes in the current demand from the AI workload, are known as voltage droops . Their magnitude depends on how much the current changes, and on di/dt, the rate of change of the current. All else equal, larger di/dt causes a larger voltage droop; conversely, the same absolute change in current over a longer period of time will result in lower di/dt and thus less droop. These droops are usually not a problem. However, all chips have a minimum voltage (Vmin) below which the chip ceases to operate normally causing incorrect results. Figure 2. Voltage droop occurs when a chip quickly draws power, discharging decoupling capacitors and quickly decreasing voltage, before the voltage regulator recovers To reduce the likelihood of this worst-case scenario, chips are operated with a voltage guardband, providing enough supply voltage margin so the chip still receives its required minimum voltage under transient and worst-case conditions. Most of the time, the power needed to provide this guardband is effectively excess. Moreover, power is proportional to the square of voltage, so 10% more voltage continually supplied 21% more power. How does NVIDIA Groq 3 LPX cycle-exact schedule make current demand predictable? The Groq 3 LPX deterministic execution model allows the creation of a schedule for how an AI workload will run, including both operations and data movement, prior to beginning the workload. Individual LPU accelerators feature a relatively small number of distinct hardware elements, each of which enables predictably fast execution of the most common operations: MXM for matrix multiplication VXM for vector operations SXM for transposing and reshaping Individual accelerators also feature hardware that keeps these compute units synchronized down to the clock cycle. For memory, each chip has hierarchy-free on-chip SRAM banks. At the LPX system level, LPUs are connected directly to each other rather than through an intermediary, making data transfer times more predictable. Determinism ties these various hardware elements together: individual computations, memory reads, and interchip communications require the same number of clock cycles from run to run. The compiler can exploit this to plan a schedule of when data will need to move between these compute elements, arriving just where it needs to, when it needs to. This schedule of compute and data movement also allows the compiler to resolve common resource conflicts, such as two hardware elements writing to the same memory bank, in advance. Figure 3. Groq 3 LPX execution model, and what its compiler emits, compared to a dynamically scheduled accelerator This cycle-exact schedule also allows the creation of the workload’s current demand over time. The compiler can estimate how much current the system will draw, at the clock cycle level. Figure 4. Groq 3 LPX cycle-exact schedule, produced by its compiler before an AI workload runs. The current demand curve can be produced from the schedule How does the execution model enable proactive voltage and clock control? The cycle-exact schedule and resulting current-demand curve enable Groq 3 LPX to reduce voltage droop and operate with a smaller voltage guardband. Because the compiler knows when current demand will rise and fall, down to the clock cycle, it can prepare the power-delivery system in advance and shape the sharpest changes in demand through the complementary PEP and CPS technologies. PEP is the mechanism through which the chip commands the power delivery network (PDN)—the physical electronics that deliver power to the LPUs—to change the voltage it delivers. Because the compiler knows which cycle current will spike on, it can schedule that command early enough that the supplied voltage is already moving before the demand arrives. The decoupling capacitors thus have less of a gap to make up, and the chip’s voltage drops less sharply when the current ramps up. CPS enables the compiler to schedule individual clock cycles in a workload to be shortened or lengthened. The cycle-by-cycle scheduling is only possible because of the Groq 3 LPX plesiosynchronous clock system, which keeps clocks synchronized between chips and compute elements synchronized within chips. It allows for more fine-grained control over the sharpest changes in current than PEP. In particular, the clock cycles with the very highest spikes in current can be lengthened, lowering di/dt, the rate at which current increases. Figure 5. PEP and CPS lead to less required voltage guardband, and thus power savings, by shaping the supply and demand for current, respectively, to allow core voltage to run closer to Vmin Together, PEP and CPS give the compiler tools to schedule electrical signals and even clock-cycle length in ways that lead to less voltage droop. Internal testing on Groq 3 LPX systems has shown that this approach leads to >60% less voltage drop. It is estimated that these will lead to a high single-digit percentage decrease in the baseline voltage the electrical system must continually provide to AI workloads. This will contribute to even higher percentage decreases in power because power is proportional to the square of voltage—all without impacting the workload. Figure 6. A smaller voltage guardband lowers the supplied voltage, and the power saved scales with the square of that reduction on every cycle Delivering more performance per watt—at high interactivity Groq 3 LPX deterministic execution can reduce the power required to run the same workload by a potentially low-double-digit percentage compared with a similarly specified, nondeterministic system. In a power-limited AI factory, lowering that overhead leaves more of the fixed power budget available to produce tokens. Groq 3 LPX brings these determinism-enabled power controls to the NVIDIA Vera Rubin platform in H2 2026. They complement NVIDIA DSX MaxLPS at the factory level and Intelligent Power Smoothing within the Vera Rubin NVL72 rack. Each operates at a different level of the platform, but all serve the same goal: getting more useful AI inference from every megawatt. At the platform level, pairing Groq 3 LPX with Vera Rubin NVL72 enables up to 35x higher throughput per megawatt compared to previous-generation NVIDIA GB200 NVL72 for 2T+ parameter models at long context and high interactivity. For AI factories, that means serving far more tokens within the same power budget at that demanding operating point. Acknowledgments This work was made possible through the expertise and engineering contributions of Ashraf Essea, Kibibi Moseley, Graham Steele, Suhas Somnath, Sarah McKenney, Farshad Ghodsian, and Eduardo Alvarez.