Liquid AI Releases QAD Q4_0 Checkpoints for LFM2.5 Models

Liquid AI Releases QAD Q4_0 Checkpoints for LFM2.5 Models Liquid AI has released 4-bit QAD Q4_0 GGUF checkpoints for its LFM2.5 model series, using Quantization-Aware Distillation to retain near-full-precision quality at standard 4-bit memory footprints and speeds. QAD distills a high-precision teacher model into a quantized student, recovering approximately 97% of the average BF16 baseline performance across four model sizes. Available model checkpoints include LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B. On benchmarked edge hardware, the smaller models match Q5_K_M quality with 4–33% faster decode throughput, while larger models match Q4_K_M quality with 3–14% higher throughput. The GGUF files are hosted on Hugging Face and can be run using llama.cpp or any GGUF Q4_0-compatible runtime. Quantization-Aware Distillation and Accuracy The release introduces QAD Q4_0 checkpoints for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B. By distilling a high-precision teacher into a quantized student model, the checkpoints maintain native Q4_0 memory and throughput while recovering 97% of the BF16 accuracy lost during standard quantization. Across evaluations spanning GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, BFCLv4, GSM8K, and AIME25, the four models retained 97.1%, 96.5%, 97.4%, and 96.6% of their BF16 baseline performance across five repeats. Edge Hardware Performance and Deployment Decode throughput was evaluated on MacBook Pro and NucBox EVO-X2 using GPU inference, as well as Samsung Galaxy S26 Ultra and Raspberry Pi 5 using Arm CPU inference. The 230M and 350M QAD checkpoints matched Q5_K_M quality within evaluation variance at 4–33% higher decode throughput. The 1.2B and 2.6B models matched Q4_K_M quality at 3–14% higher throughput, while also matching Unsloth's UD-Q4_K_XL where applicable. The checkpoints are available on Hugging Face and can be executed directly using llama.cpp or compatible runtimes.

Liquid AI发布基于量化感知蒸馏的LFM2.5 Q4_0检查点 Liquid AI宣布推出采用量化感知蒸馏（QAD）技术的LFM2.5系列Q4_0 GGUF模型检查点，涵盖230M、350M、1.2B-Instruct和2.6B四种规模，在保持原生Q4_0内存占用与速度的同时大幅挽回精度损失。 QAD技术通过高精度教师模型向低精度学生模型蒸馏，挽回了因量化损失的97% BF16平均精度。 评测显示四款模型的QAD检查点分别保留了BF16基线97.1%、96.5%、97.4%和96.6%的性能。 在MacBook Pro、NucBox EVO-X2、三星Galaxy S26 Ultra及树莓派5上，新检查点以更高吞吐量匹敌更高规格量化版本的质量。 相关GGUF文件已上线Hugging Face，兼容llama.cpp等支持Q4_0的运行时。 量化感知蒸馏与性能表现 针对LFM2.5-230M、LFM2.5-350M、LFM2.5-1.2B-Instruct与LFM2.5-2.6B，Liquid AI使用高精度教师模型向量化学生模型进行QAD蒸馏。在包含GPQA Diamond、MMLU-Pro、IFEval等多项基准及数学评测（GSM8K或AIME25）的测试中，QAD检查点大幅优于传统训练后量化（PTQ），分别保留了各自BF16基线97.1%、96.5%、97.4%和96.6%的性能表现。 边缘硬件吞吐量与生态兼容 在涵盖GPU（MacBook Pro、NucBox EVO-X2）与Arm CPU（三星Galaxy S26 Ultra、树莓派5）的边缘硬件测试中，230M和350M的QAD Q4_0在解码吞吐量提升4-33%的同时达到Q5_K_M级质量；1.2B与2.6B版本则在吞吐量提升3-14%的情况下匹敌Q4_K_M质量，并与Unsloth的UD-Q4_K_XL相当。该系列GGUF文件支持直接通过llama.cpp运行。

Today, we release QAD Q4_0 GGUFs. These are updated 4-bit checkpoints for LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B. They allow developers to run LFM2.5 models at Q4_0 memory and speed without the usual quality drop: Trained with Quantization-Aware Distillation (QAD) : a high-precision teacher model is distilled into a quantized student model Same memory and speed as native Q4_0: They keep the low memory footprint and high throughput of Q4_0 GGUFs Recovery: 97% of their BF16 average accuracy lost to quantization is recovered Benchmark results For all four models, we compare their released GGUFs produced with post-training quantization (PTQ) against the trained QAD Q4_0 checkpoints on a benchmark suite spanning reasoning, instruction-following, tool use, and agentic capabilities: GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, and BFCLv4. The BF16 GGUF serves as the in-format ceiling. We also add one scale-appropriate math evaluation: GSM8K for LFM2.5-230M and LFM2.5-350M, and AIME25 for LFM2.5-1.2B-Instruct and LFM2.5-2.6B. We report the mean across five repeats. Across all four models, QAD substantially improves the Q4_0 checkpoint. The QAD checkpoints retain 97.1%, 96.5%, 97.4%, and 96.6% of their respective BF16 baseline performance. Speed and size on real edge hardware We measure decode throughput for the four models LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B across four targets: MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5. MacBook Pro and NucBox use GPU inference, while Samsung and Raspberry Pi use Arm CPU inference. BF16 and F16 are shown as full-precision references where profiled. The 230M and 350M QAD Q4_0 checkpoints match Q5_K_M quality within evaluation variance at a 4-33% higher decode throughput. The 1.2B and 2.6B QAD Q4_0 checkpoints match Q4_K_M quality at a 3-14% higher throughput. The QAD Q4_0 checkpoints also match Unsloth's UD-Q4_K_XL (where applicable, for the 230M and 1.2B), a strong external post-training quantization checkpoint. How to use QAD GGUFs Use the files with llama.cpp or any runtime that supports GGUF Q4_0 artifacts. llama-cli -hf LiquidAI/LFM2.5-350M \ --hf-file LFM2.5-350M-QAD-Q4_0.gguf \ -p "What is C. elegans?" Get Started with QAD GGUFs The QAD GGUFs are available on Hugging Face today: LFM2.5-230M , LFM2.5-350M , LFM2.5-1.2B-Instruct , and LFM2.5-2.6B . We can't wait to see what you build. Citation For citations, please use the following reference or BibTeX: Liquid AI, "LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment", Liquid AI Blog, Aug 2026. Or use the BibTeX citation @article{liquidAI2026Q40, author = {Liquid AI}, title = {LFM2.5 Q4_0: Quantization-Aware Distillation for Edge Deployment}, journal = {Liquid AI Blog}, year = {2026}, note = {www.liquid.ai/blog/qad}, }