Quantization-Aware Healing Enables 4-Bit Models to Outperform Full-Precision Baselines

Quantization-Aware Healing Enables 4-Bit Models to Outperform Full-Precision Baselines Researchers at Multiverse Computing have introduced Quantization-Aware Healing (QAH), a method that distills a compressed 4-bit model directly from its original uncompressed teacher rather than from intermediate checkpoints. Applying QAH to a 60B model derived from GPT-OSS 120B produced a 4-bit MXFP4 model that outperformed its recovered bfloat16 baseline on seven out of nine benchmarks. QAH distills directly from the original full-size, full-precision model using KL divergence, bypassing intermediate degraded checkpoints. A 60B MXFP4 model trained with QAH beat its 60B bfloat16 version on 7 of 9 benchmarks, showing the largest gains in long-context reasoning (+7.4) and math (+5.6). In a head-to-head comparison on a GPT-OSS 9B model, QAH converged to its peak performance roughly 7 times faster than quantization-aware training (QAT) and avoided post-peak degradation. QAH utilizes a memory-efficient chunked KL-divergence loss to support long-context healing sequences of up to 32k tokens on fixed GPU memory budgets. Limitations of Existing Healing Pipelines Standard compression workflows cut model architectures (removing layers, heads, or neurons) and quantize weights to 4 bits, which degrades core reasoning and coding capabilities. Conventional quantization-aware training (QAT) relies on task-loss fine-tuning across a noisy low-precision forward pass, which is costly and prone to post-peak degradation. Meanwhile, standard quantization-aware distillation (QAD) is typically anchored to an already compressed and degraded intermediate bfloat16 checkpoint, capping student performance at that lower ceiling. Architecture-Agnostic Distillation Directly from the Original Model QAH overcomes these limits by utilizing the original pre-compression, full-precision model as the frozen teacher, matching its logit distribution via KL-divergence loss. Because output distributions are architecture-agnostic, the mismatch in size and precision does not disrupt transfer, turning quantization into an additional knowledge transfer stage. To accommodate contexts up to 32k tokens, QAH incorporates a chunked KL loss that evaluates sequence slices without materializing the entire vocabulary-by-sequence grid. Benchmark Performance and Training Stability Applied to a GPT-OSS 120B model compressed to 60B parameters in MXFP4, QAH matched or exceeded the 60B bfloat16 version on seven of nine benchmarks, trailing by less than 1.5 points only on MMLU-Pro and SciCode, while outperforming the original 120B teacher on LiveCodeBench (66.5 vs. 66.0). In stability tests against QAT on a 9B model, QAH peaked around step 100 compared to step 700 for QAT; QAH maintained its accuracy thereafter, whereas QAT dropped nearly 19 points by step 1,200.

Multiverse Computing推出QAH修复方法：4比特压缩模型性能反超全精度原版 Multiverse Computing提出了“量化感知修复”（QAH）方法，解决了大语言模型经结构压缩与4比特量化后能力大幅下降的问题。在对GPT-OSS 120B压缩至60B并量化为MXFP4的实验中，所得4比特模型在9项基准测试中有7项超越了其全精度bfloat16版本。 QAH改变了传统修复逻辑，让量化学生模型直接通过KL散度蒸馏未经压缩的全尺寸原始教师模型，而非依赖结构压缩后退化的恢复检查点。 在GPT-OSS 60B MXFP4模型测试中，QAH版本在7/9基准上优于其60B BF16检查点，尤其在长上下文推理（AA-LCR +7.4）和数学（AIME 2025 +5.6）上提升明显，甚至在LiveCodeBench上超越120B教师模型。 对比传统QAT方法，QAH在GPT-OSS 9B实验中仅需约100步即达峰值（QAT需约700步），且后续训练表现稳定，避免了QAT过训练导致的断崖式崩溃（QAT在1200步衰减近19分）。 QAH采用分块KL散度损失以支持32k长上下文训练，在降低显存占用与计算开销的同时提高了模型精度。 传统修复方法的瓶颈与QAH机制 现有的量化感知训练（QAT）需要重新运行昂贵的后训练流程且容易不稳定，而传统量化感知蒸馏（QAD）在模型经历剪枝或结构压缩后，只能以退化的全精度恢复检查点为师，受限于其性能上限。QAH打破了这一假设，直接将全尺寸、全精度的原始教师模型logits通过KL散度蒸馏至仅有一半参数且处于MXFP4格式的学生模型。量化阶段由此转变为第二轮蒸馏学习，借此补全结构压缩阶段未能转移的信息。 基准测试结果与精度反超 在将GPT-OSS 120B压缩至60B并量化为MXFP4的测试中，QAH模型在7项基准上击败了其最佳全精度版本60B BF16。提升最显著的领域正是压缩通常损害最深的能力：AA-LCR提升7.4分，AIME 2025提升5.6分。仅落后的MMLU-Pro和SciCode分差均在1.5分以内。此外，该4比特模型在LiveCodeBench上以66.5分超越了120B教师模型的66.0分，GPQA Diamond也达到67.4分，紧追教师的69.0分。 训练稳定性对比与实际部署优势 在GPT-OSS 9B的同条件测试中，QAH与QAT峰值相近（54.9对54.6），但QAH达到峰值的速度快约7倍（约100步比700步），并在随后训练中维持在峰值2分范围内。相比之下，QAT越过峰值后在1200步暴跌近19分。QAH通过固定教师分布消除了过度推动硬标签导致的漂移风险。结合分块KL损失处理长达32k token的长上下文，4比特QAH模型相比bfloat16学生节省约4倍权重显存，并在较小硬件上实现了更优的精度与更低的推理成本。

Making a large language model smaller almost always comes with a cost. The now-standard recipe for efficient deployment is to compress the architecture first, cutting the parameter count by removing layers, heads, or neurons, and then quantize the remaining weights down to 4 bits to shrink memory and compute further. Both steps save a lot, but together they systematically degrade the capabilities people actually care about: reasoning, mathematical problem-solving, and code generation. Because of this, serious deployment pipelines add a recovery step, usually called healing, before the model goes into production. Recent open-weight releases such as gpt-oss , NVIDIA's Nemotron family, and our own Hypernova 60B all rely on some version of this compress-then-heal approach. Our latest paper, Quantization-Aware Healing: A Practical Recipe for Recovering Compressed, 4-Bit LLMs , asks a question that the field has mostly left open: once a model has already been through structural compression, not just quantization, how well does that recovery step actually work, and what is the right way to do it? We introduce Quantization-Aware Healing (QAH), and applied to a GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4, it produces a model that beats its own full-precision (bfloat16) version on 7 of 9 benchmarks. The 4-bit model ends up smaller, cheaper to run, and more accurate than the checkpoint it was quantized from. This inverts the usual relationship between a 4-bit model and the 16-bit model it came from. Why the usual healing methods fall short here Most efficiency pipelines follow the same three steps: compress the architecture, quantize the compressed weights, then heal the damage. The difference between methods is entirely in that last step. The dominant healing recipe is quantization-aware training (QAT). It inserts fake-quantization operators into the forward pass and keeps fine-tuning the model on a task loss, so the weights learn to tolerate the low-precision representation. In practice this means re-running an already expensive multi-stage post-training process, supervised fine-tuning, RLHF, agentic tuning, through a noisier, lower-precision forward pass. It is costly, and as our results show, it can also become unstable if training continues too long past its best point. An alternative, quantization-aware distillation (QAD), avoids re-running that history. Instead of a task loss, it distills a frozen full-precision teacher directly into the quantized student through a KL-divergence loss on the output logits. This works well when the only change is quantization, because a genuine full-precision version of the exact same model exists to act as teacher. But once a model has gone through structural compression, fewer layers, heads, or neurons, and not just fewer bits, that assumption breaks. There is no independently trained full-precision version of the smaller architecture. The only candidate teacher is the recovered bfloat16 checkpoint, which is itself a distilled approximation of the original model. Distilling from it anchors the quantized student to a degraded target and caps its accuracy at that recovered checkpoint's own ceiling. So the question of how to heal a model that has been both structurally compressed and quantized was, until now, genuinely open. Our approach QAH removes that ceiling with one change: it distills directly from the original, pre-compression model rather than from the recovered one. Teacher and student do not even share an architecture. The teacher is full-size and full-precision, the student is half the size and running in MXFP4. Because a teacher's output distribution is architecture-agnostic, nothing about the size or shape mismatch prevents the transfer. The student never sees hard labels, only the teacher's output distribution, matched through KL divergence on the logits. This reframes what the quantization stage is doing. Under QAH it is no longer a lossy postprocessing step applied after healing is finished. It is a second, full pass of distillation against the original teacher, supervision that the bfloat16 checkpoint never received. The 4-bit student is not compensating for information lost to quantization; it is picking up information the earlier recovery stage did not have the time or data to transfer. There is also a stability benefit that falls out of the loss itself. Because KL distillation ties the student to a fixed teacher distribution, once the student catches up there is no further pressure for it to drift. A cross-entropy task loss, by contrast, keeps pushing the student toward hard labels indefinitely. That difference turns out to matter for both accuracy and training stability, as the comparison below shows. To make QAH work at long context, where the healing corpus includes documents up to 32k tokens, we reuse the memory-efficient chunked KL-divergence loss from our companion paper on efficient distillation. That loss computes the KL one slice of the sequence at a time and never materializes the full vocabulary-by-sequence grid, which is what makes 32k-token healing fit inside a fixed GPU memory budget. We covered the mechanics of that loss in a previous post . QAH overview. After structural compression and quantization, capabilities drop sharply. QAH distills from the original model, a frozen teacher whose logits are precomputed offline, rather than from the recovered checkpoint. Source: paper Figure 1. Results We applied QAH to a GPT-OSS 120B model, compressed to 60B parameters and recovered in bfloat16, then re-quantized to MXFP4 under QAH. The natural comparison is against that same 60B model's bfloat16 checkpoint, the best full-precision version of this architecture that exists. The QAH model wins on 7 of the 9 benchmarks. Benchmark 120B teacher (MXFP4) 60B BF16 (recovered) 60B MXFP4 (QAH) QAH vs BF16 AA-LCR (long-context reasoning) 50.0 35.3 42.7 +7.4 AIME 2025 (math) 80.0 70.7 76.3 +5.6 Aider (agentic coding) 45.3 38.2 40.9 +2.7 τ²-bench (tool use) 68.4 59.4 61.7 +2.3 GPQA Diamond (science) 69.0 65.7 67.4 +1.7 IFBench (instruction following) 63.3 58.4 59.9 +1.5 LiveCodeBench (coding) 66.0 65.5 66.5 +1.0 MMLU-Pro (knowledge) 78.0 74.0 73.8 −0.2 SciCode (science coding) 37.5 35.6 34.2 −1.4 The two benchmarks where QAH trails, MMLU-Pro and SciCode, lose by less than a point and a half. Everywhere else the 4-bit model is ahead of its own 16-bit source, and the largest gains land on exactly the capabilities compression usually damages most: long-context reasoning (+7.4 on AA-LCR) and math (+5.6 on AIME 2025). The comparison against the original 120B teacher is just as telling. Despite running at half the teacher's parameter count and roughly a quarter of its weight memory, the QAH model surpasses the full-size teacher on LiveCodeBench (66.5 vs. 66.0) and comes within 1.6 points on GPQA Diamond (67.4 vs. 69.0). The largest remaining gap against the teacher is on AA-LCR, an extreme long-context benchmark where the capacity lost to compression is intrinsically the hardest to recover. The 4-bit QAH model matches or beats its bfloat16 source on 7 of 9 benchmarks, and beats the full-size teacher on LiveCodeBench. Source: paper Figure 2. QAH against QAT, head to head To isolate the effect of the loss function from everything else, we also compared QAH directly against QAT under matched conditions, quantizing a GPT-OSS 9B model to MXFP4 and tracking average performance across MMLU-Pro, LiveCodeBench, and GPQA Diamond as training progresses. Both methods reach a similar peak, 54.9 for QAH against 54.6 for QAT, so on best-case accuracy they are effectively tied. The difference is in how they get there and what happens afterwards. QAH reaches its peak in about 100 steps, roughly 7 times faster than QAT's 700, and then stays within about two points of that peak for the rest of training. QAT collapses sharply once past its peak, shedding nearly 19 points by step 1,200. The practical consequence is a real deployment risk difference. A QAT checkpoint needs careful early stopping against a held-out signal to avoid shipping a model that has already started to degrade, whereas a sufficiently trained QAH checkpoint can be served safely because it simply does not drift. This is consistent with the mechanism: KL distillation against a frozen teacher gives the student no incentive to move once it matches the teacher, while a cross-entropy objective keeps pushing on hard labels and eventually erodes capabilities the model inherited from the original. QAH peaks at 54.9 in roughly 100 steps and holds; QAT reaches 54.6 only around step 700, then loses nearly 19 points by step 1,200. Source: paper Figure 3. What this changes in practice The accuracy story comes paired with the efficiency story that motivated compression in the first place. At 4-bit precision the QAH model uses roughly 4 times less weight memory than the bfloat16 student, and at half the parameter count of the 120B teacher it roughly halves compute per token, which is what lets it run on substantially smaller hardware. For model families that ship in bfloat16 rather than 4-bit, the combined parameter and precision reduction would be closer to 8 times less compute per token. The takeaway is that a compressed, 4-bit model does not have to be a lower-accuracy version of its full-precision counterpart. With this healing recipe it can be smaller, cheaper to serve, and more accurate at the same time, and it reaches that point in a fraction of the training a QAT recipe would need. Quantization stops being a tax you pay for efficiency and becomes an extra opportunity to teach the model. This work is part of Multiverse Computing's ongoing research into making large models smaller and cheaper to run without giving up the capabilities that make them useful. It sits alongside our companion work on efficient distillation, which supplies the long-context training machinery QAH depends on. Want the full technical details, including the healing pipeline, the chunked KL implementation for long-context healing, and the distributed-training findings? Read the full paper , or get in touch with our team to talk about applying compression and healing to your own models.