Liquid AI Releases LFM2.5-DSpark Draft Models for Accelerated Inference

Liquid AI Releases LFM2.5-DSpark Draft Models for Accelerated Inference Liquid AI has released DSpark draft model checkpoints for three LFM2.5 models, using speculative decoding to deliver up to a 3.18x throughput increase on GPUs and up to 2.87x on edge devices without altering greedy output quality. Draft models around 300M parameters were released for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. Throughput improves by up to 3.18x on an H100 GPU and 2.87x on an M4 Max MacBook Pro, while cutting function-calling latency by 57% on average for LFM2.5-2.6B. Speculative verification guarantees greedy decoding output remains identical to the baseline target model. The models ship with day-one upstream support for llama.cpp and SGLang in both Safetensors and GGUF formats. DSpark Architecture and Quality Parity LLM decoding is typically memory-bound, spending substantial latency loading weights from DRAM to SRAM. DSpark addresses this via speculative decoding by combining a DFlash-style parallel backbone conditioned on target features, a lightweight sequential Markov head to raise later-position acceptance, and a confidence-scheduled verifier to prune unpromising suffixes. The resulting models are simplified 5-layer, attention-only architectures with a block size of 9 and approximately 300M parameters. Under greedy decoding, proposed tokens are verified against the target model distribution; any rejection falls back to the target model's own token, ensuring emitted text is identical by construction to baseline greedy output. Inference Benchmarks Across GPU and Edge Tested on an Nvidia H100 in BF16 and an Apple M4 Max MacBook Pro in FP16 GGUF across five benchmark datasets, the draft models achieved substantial throughput gains. LFM2.5-2.6B saw average speedups of 2.67x on the H100 (reaching 864 tokens per second) and 2.27x on the M4 Max (reaching 139 tokens per second). LFM2.5-1.2B-Instruct averaged 2.10x on H100 and 2.54x on the MacBook. The MoE model, LFM2.5-8B-A1B, achieved a 2.54x average speedup on the H100 but only an 18% improvement on the MacBook due to the Metal backend in llama.cpp activating more experts and generating higher weight traffic during multi-token verification. Software Integration and Deployment Day-one support is open-sourced upstream for both SGLang and llama.cpp. Users can serve models through an OpenAI-compatible endpoint using SGLang with flashinfer backends, or run local instances via llama-server using GGUF weights with Metal acceleration. The checkpoints are distributed on Hugging Face in Safetensors and GGUF formats across all three model variants.

Liquid AI 发布 LFM2.5-DSpark 推理加速草稿模型，GPU 吞吐提升最高达 3.18 倍 Liquid AI 为旗下 LFM2.5 系列模型发布了 DSpark 投机解码草稿模型检查点，在不改变贪心解码输出质量的前提下，实现 GPU 最高 3.18 倍、端侧最高 2.87 倍的推理加速，并首日支持 SGLang 与 llama.cpp。 为 LFM2.5-1.2B-Instruct、LFM2.5-2.6B 及 LFM2.5-8B-A1B 三款模型提供约 300M 参数的 DSpark 草稿模型。 采用基于目标模型特征的 DFlash 式并行主干、轻量马尔可夫头与置信度调度验证器，且贪心输出与基准模型完全一致。 在 H100 与 M4 Max MacBook Pro 上的测试中均实现明显加速，同时平均降低 LFM2.5-2.6B 函数调用延迟 57%。 提供 Safetensors 和 GGUF 格式权重，并在 SGLang 和 llama.cpp 中开放了上游集成支持。 DSpark 架构与质量一致性 DSpark 针对 LLM 解码阶段显存受限的瓶颈，通过草稿模型预生成候选 token，再由目标模型单次前向完成统一验证。其架构结合了生成隐藏状态的并行主干、捕捉相邻 token 依赖的序列马尔可夫头，以及修剪低置信度后缀的调度验证器。模型仅含 5 层注意力结构，规模约 300M 参数。在贪心解码下，被拒 token 会被目标模型自身预测替代，生成序列与基准模型完全相同，评测基准准确率保持不变。 跨硬件基准测试表现 评测基于单张 H100（BF16、SGLang）和搭载 M4 Max 的 MacBook Pro（FP16 GGUF、llama.cpp Metal），批大小为 1，块大小为 9。LFM2.5-2.6B 在 H100 上平均加速 2.67 倍，在 MacBook 上平均达 2.27 倍（吞吐达 139 tok/s），并在多工具场景下将函数调用延迟降低 57%。LFM2.5-1.2B-Instruct 在端侧表现更优，平均加速 2.54 倍；而 MoE 模型 LFM2.5-8B-A1B 在 H100 上最高达 3.18 倍，但端侧仅提升 18%，受限于当前 llama.cpp 的 Metal MoE 实现及多 token 验证触发的专家权重流量开销。 部署与开源可用性 相关权重已在 Hugging Face 提供 Safetensors 和 GGUF 格式。SGLang 已通过 PR #31041 支持附加草稿模型运行，用户可配置 flashinfer 后端并直接启动兼容 OpenAI 的推理服务接口；llama.cpp 则通过 PR #27383 支持以 sidecar 元数据结合实验性 Metal 内核在端侧运行。

Today, we release DSpark draft model checkpoints for three models from our LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality: Faster inference : up to 3.18 throughput improvement on a GPU and up to 2.87x on-device. Toward on-device agentic inference : cuts function-calling latency by 57% on average for LFM2.5-2.6B Day-one support for llama.cpp and SGLang : LFM-compatible DSpark integration is open-sourced upstream How does DSpark work The decode phase in LLM inference is traditionally memory-bound. Most latency comes from streaming weights from DRAM into SRAM, not from intense computation. Speculative decoding addresses this by using a lightweight draft model to produce candidate tokens, then having the target model verify them all in a single forward pass, sharing the cost of loading the weights across all tokens we verify. Over the years, multiple approaches of speculation have been proposed, with the most prominent being EAGLE-3 , DFlash , and, most recently, DSpark , which combines three components: DFlash-style parallel backbone conditioned on the target model’s context features, producing hidden states for all draft tokens in a single forward pass. A lightweight sequential head , modeled as a Markov chain between neighboring tokens, that adds inter-token dependency, raising the acceptance rate at later positions. A confidence-scheduled verifier that predicts each token’s survival probability and prunes low-confidence suffixes when verification would cost more than it saves. Training and Architecture We follow the DSpark recipe with a larger and more diverse data mix covering SFT, chat, code, and function-calling data. Based on our ablations, the first versions of the draft models are simplified attention-only draft models, with 5 layers and a block of 9. For each draft model, we ran 15 epochs on the entire dataset and selected the epoch with the highest acceptance rate rather than the lowest loss. The resulting draft models are relatively small, with each around ~300M parameters. Component LFM2.5-1.2B-Instruct LFM2.5-8B-A1B LFM2.5-2.6B Decoder stack (5 layers) 241.2M 241.2M 241.2M Hidden-state projection 21.0M 21.0M 21.0M Markov head 33.6M 65.5M 65.5M Norms + confidence head 27.5k 27.5k 27.5k Total 295.7M 327.7M 327.7M Quality parity Under greedy decoding, a draft token is only accepted if it matches the target model’s distribution. On rejection, the target model's own token takes its place. The emitted sequence is therefore identical to baseline greedy by construction, so benchmark accuracy (pass@1 or exact match) is unchanged. Inference Speed Up on CPU and GPU Our DSpark draft models for LFM2.5 ship with day-one support for llama.cpp (implementation builds on top of the official codebase , which we run with experimental metal kernels ) and **SGLang (**implementation builds on the official SGLang implementation of DSpark ). We measure on-device throughput with llama.cpp and Metal on an M4 Max MacBook Pro using FP16 GGUF weights and up to 256 output tokens. We measure GPU throughput with SGLang on a single H100 80 GB in BF16. Both configurations use a DSpark block size of 9, a batch size of 1, and a temperature of 0. We evaluate them on five benchmark datasets. All three drafter models deliver noticeable throughput improvements on both the large-scale accelerator (H100) and the edge deployment (M4 Max MacBook). For LFM2.5-2.6B, speedup on the MacBook is especially noticeable, as it pushes the interactivity level a user can enjoy far beyond the throughput offered by most proprietary cloud models (around ~140 tok/s, depending on the dataset). Dataset Acceptance (of 10) Speedup on H100 Speedup on M4 Max MATH500 5.42 3.06x 326 → 1000 tok/s 2.25x 61 → 137 tok/s HumanEval 4.54 2.56x 326 → 835 tok/s 2.63x 61 → 161 tok/s MBPP 4.71 2.64x 326 → 861 tok/s 2.11x 62 → 132 tok/s GSM8K 4.32 2.22x 312 → 693 tok/s 2.36x 60 → 143 tok/s MT-Bench 5.07 2.87x 325 → 933 tok/s 1.99x 62 → 123 tok/s Mean 4.81 2.67x 323 → 864 tok/s 2.27x 61 → 139 tok/s Across various multi-tool scenarios, DSpark reduces the latency by 57% on average for LFM2.5-2.6B. For LFM2.5-1.2B-Instruct , we see much more variance in dataset acceptance rates, so speedup varies by as much as 52% depending on the underlying text distribution. Dataset Acceptance (of 10) Speedup on H100 Speedup on M4 Max MATH500 6.02 2.56x 668 → 1712 tok/s 2.62x 140 → 366 tok/s HumanEval 5.31 2.26x 664 → 1499 tok/s 2.87x 136 → 389 tok/s MBPP 5.52 2.37x 667 → 1578 tok/s 2.74x 137 → 375 tok/s GSM8K 4.34 1.67x 624 → 1041 tok/s 2.73x 140 → 381 tok/s MT-Bench 3.90 1.66x 657 → 1091 tok/s 1.72x 137 → 237 tok/s Mean 5.02 2.10x 656 → 1384 tok/s 2.54x 138 → 350 tok/s For LFM2.5-8B-A1B , the acceptance rate increases compared to two dense models, yet on-device we get only an 18% improvement on average. This gap is due to the current MoE implementation in llama.cpp's Metal backend, and to the fact that verifying k tokens activates more experts and thus more weight traffic than a single decode step. Dataset Acceptance (of 10) Speedup on H100 Speedup on M4 Max MATH500 8.27 3.18x 428 → 1362 tok/s 1.21x 93 → 112 tok/s HumanEval 7.02 2.58x 426 → 1100 tok/s 1.12x 91 → 101 tok/s MBPP 6.93 2.64x 426 → 1122 tok/s 1.09x 89 → 97 tok/s GSM8K 4.02 1.29x 385 → 496 tok/s 1.44x 90 → 129 tok/s MT-Bench 8.52 3.02x 426 → 1288 tok/s 1.04x 87 → 90 tok/s Mean 6.95 2.54x 418 → 1074 tok/s 1.18x 90 → 106 tok/s How to use LFM2.5-DSpark Running the DSpark draft models with SGLang requires an SGLang build with DSpark support for LFM2 targets ( PR #31041 ). Launch the target with the draft attached: python -m sglang.launch_server \ --model-path LiquidAI/LFM2.5-2.6B \ --speculative-algorithm DSPARK \ --speculative-draft-model-path LiquidAI/LFM2.5-2.6B-DSpark \ --speculative-draft-attention-backend flashinfer \ --disable-radix-cache --mem-fraction-static 0.75 --port 30000 Then query the OpenAI-compatible endpoint at http://localhost:30000/v1 . The block size is read from the draft's config.json ; the baseline is the same command without the three --speculative-* flags. Running them with llama.cpp requires the respective llama.cpp build ( PR#27383 ). llama-server -m LFM2.5-2.6B-F16.gguf \ -md LFM2.5-2.6B-DSpark-F16.gguf \ --spec-type draft-dspark --spec-draft-n-max 10 --spec-draft-n-min 0 \ -fa on -ngl 99 The block size is read from the sidecar metadata (n-max is clamped to it). Speculative decoding is exact : the target verifies every proposed token, so greedy output equals the target alone; per-response timings report draft_n / draft_n_accepted . Get Started The DSpark draft model checkpoints are available on Hugging Face as Safetensors and in GGUF format: Safetensors : LFM2.5-2.6B-DSpark , LFM2.5-1.2B-Instruct-DSpark , and LFM2.5-8B-A1B-DSpark GGUF : LFM2.5-2.6B-DSpark-GGUF , LFM2.5-1.2B-Instruct-DSpark-GGUF , LFM2.5-8B-A1B-DSpark-GGUF We can’t wait to see what you build. Citation For citations, please use the following reference or BibTeX: Liquid AI, "LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook", Liquid AI Blog, Aug 2026. @article{liquidAI2026dspark, author = {Liquid AI}, title = {LFM2.5-DSpark: Up to 3.2x Faster Inference from H100 to MacBook}, journal = {Liquid AI Blog}, year = {2026}, note = {www.liquid.ai/blog/lfm2.5-dspark}, }