Up To 3.2X Faster Inference With LFM2.5-DSpark

LiquidAI has released DSpark draft model checkpoints for three models in its LFM2.5 family — LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B — adding a speculative decoding path that the company says delivers up to 3.18x faster inference on a GPU and up to 2.87x on-device with a minimal memory increase and no change to output quality. The release ships with day-one support for llama.cpp and SGLang, and the LFM-compatible DSpark integration has been open-sourced upstream, the company said.

DSpark is a speculative decoding technique that tackles the memory-bound nature of LLM decoding. Most decode-phase latency comes from streaming model weights from DRAM into SRAM rather than from computation, LiquidAI explained. A lightweight draft model — around 300 million parameters per checkpoint — proposes candidate tokens, and the target model verifies them all in a single forward pass, sharing the cost of loading weights across multiple tokens.

According to LiquidAI, each DSpark draft model combines three components: a DFlash-style parallel backbone conditioned on the target model’s context features that produces hidden states for all draft tokens at once; a lightweight sequential Markov head that adds inter-token dependency to raise acceptance rates; and a confidence-scheduled verifier that prunes low-confidence suffixes when verification would cost more than it saves. The draft models use a simplified attention-only architecture with 5 layers and a block size of 9, trained for 15 epochs on a mix of SFT, chat, code, and function-calling data, with the final epoch selected by highest acceptance rate rather than lowest loss.

The company’s published benchmarks show the largest GPU speedup on LFM2.5-8B-A1B — 3.18x on MATH500 on an H100 (428 to 1,362 tokens per second) — and the largest on-device gain on LFM2.5-1.2B-Instruct, at 2.87x on HumanEval on an M4 Max MacBook Pro (136 to 389 tok/s). For LFM2.5-2.6B on the MacBook, LiquidAI reports roughly 140 tok/s average throughput, which it says exceeds most proprietary cloud models for interactivity. Across multi-tool scenarios, DSpark cuts function-calling latency by 57% on average for LFM2.5-2.6B, according to the company.

Faster On-Device AI Without Quality Trade-offs

The release matters because speculative decoding is one of the few inference optimizations that can deliver large speedups without changing model outputs. Under greedy decoding, LiquidAI states, a draft token is only accepted if it matches the target model’s distribution; on rejection, the target model’s own token is emitted. The result is that the output sequence is identical to baseline greedy decoding by construction, leaving benchmark accuracy such as pass@1 and exact match unchanged.

The reported on-device numbers are aimed squarely at the emerging market for local and agentic AI. A 57% reduction in function-calling latency directly affects how responsive on-device agents can be when chaining tool calls, and throughput around 140 tok/s on consumer Apple hardware would put small open models in the same latency class as hosted cloud services. For developers deploying small models at the edge or serving them cheaply on GPUs, near-3x throughput gains translate into lower serving costs or better user experience at the same cost.

From EAGLE-3 to DSpark

Speculative decoding has developed through several generations of methods, with EAGLE-3 and DFlash among the most prominent prior approaches, according to LiquidAI. DSpark, the most recent, combines the parallel draft backbone of DFlash with the sequential Markov head and confidence-based pruning. LiquidAI’s release follows the DSpark training recipe but uses what it describes as a larger and more diverse data mix.

The LFM2.5 family is LiquidAI’s current generation of small language models, including both dense models (1.2B and 2.6B parameters) and a mixture-of-experts model (8B-A1B). The three new draft checkpoints range from 295.7M to 327.7M total parameters, most of which sits in the shared 241.2M decoder stack.

“These add a speculative decoding path that trades a minimal memory increase for a large decoding speedup without changing output quality: up to 3.18x throughput improvement on a GPU and up to 2.87x on-device.”

— LiquidAI, announcement

Benchmark Limits and Uneven Gains

All performance figures are vendor-reported benchmarks and have not been independently verified. The measurements were taken under specific conditions: an H100 80GB in BF16 with SGLang for GPU results, and an M4 Max MacBook Pro with FP16 GGUF weights and experimental Metal kernels via llama.cpp for on-device results, all at batch size 1, temperature 0, block size 9, and up to 256 output tokens. Real-world workloads may differ.

The speedups also vary widely by model and dataset. For LFM2.5-1.2B-Instruct, LiquidAI itself notes that speedup varies by as much as 52% depending on the text distribution. The mixture-of-experts LFM2.5-8B-A1B shows strong GPU gains but only an 18% average improvement on-device, which the company attributes to limitations in llama.cpp’s current MoE Metal backend and the extra expert activation required when verifying multiple tokens. It is not clear when those backend limitations will be addressed, and results with sampling temperatures above zero were not reported in detail.

Deployment Path for Developers

Developers can run the draft checkpoints now. On SGLang, running DSpark requires a build with LFM-target DSpark support (PR #31041), launching the target model with the –speculative-algorithm DSPARK flag and the corresponding draft checkpoint path. On llama.cpp, the integration is open-sourced upstream and works with GGUF weights on Metal.

Looking ahead, likely developments include improvements to llama.cpp’s MoE Metal backend that could narrow the on-device gap for the 8B-A1B model, broader adoption of the upstreamed DSpark support in standard llama.cpp and SGLang releases, and additional draft-model refinements — LiquidAI describes these as the first versions of the draft models, suggesting further iterations are possible. Independent benchmarking of the checkpoints on other hardware will show whether the reported gains hold beyond Apple Silicon and NVIDIA H100 setups.

Key Questions

What is DSpark speculative decoding?

DSpark is a speculative decoding method that uses a small draft model (~300M parameters) to propose candidate tokens, which the larger target model then verifies in a single forward pass. Because weight-loading costs are shared across verified tokens, decoding becomes faster without changing the final output.

Does DSpark change model outputs or accuracy?

According to LiquidAI, no. Under greedy decoding, draft tokens are only accepted if they match the target model’s distribution, and rejected drafts are replaced by the target model’s own token. The company says output sequences are identical to baseline by construction, so benchmark accuracy is unchanged.

Which hardware was used for the reported benchmarks?

GPU results were measured with SGLang on a single H100 80GB in BF16; on-device results used llama.cpp with FP16 GGUF weights on an M4 Max MacBook Pro with experimental Metal kernels. Both used a block size of 9, batch size 1, and temperature 0.

Why is the MoE model’s on-device speedup much smaller?

LFM2.5-8B-A1B averaged only an 18% on-device improvement. LiquidAI attributes this to the current MoE implementation in llama.cpp’s Metal backend and to the fact that verifying multiple draft tokens activates more experts, increasing weight traffic compared to a single decode step.

How do developers use the DSpark checkpoints?

With SGLang, launch the server with the target model plus –speculative-algorithm DSPARK and –speculative-draft-model-path pointing to the corresponding DSpark checkpoint, using a build that includes DSpark support for LFM targets (PR #31041). The llama.cpp integration is open-sourced upstream.

Source: Hugging Face