Spark-X2.6-4B-DSpark-GGUF

Spark-X2.6-4B-DSpark-GGUF provides a BF16 GGUF draft model for speculative decoding with Spark-X2.6-4B-GGUF. The target model can use BF16 or quantized GGUF weights.

DSpark accelerates generation through speculative decoding: the draft proposes up to seven tokens per drafting pass, and the target model verifies the proposed tokens in parallel, reducing the number of sequential target decoding steps.

Model Details

Property	Spark-X2.6-4B-DSpark
Target model	Spark-X2.6-4B
Draft parameters	624M
Draft weight precision	BF16
Draft layers	5 SWA layers
Sliding-window size	512
Hidden size	2560
Draft tokens per forward pass	7
Target hidden-state layers	[2, 10, 18, 25, 33]

Performance

Mean Acceptance Length

We evaluate acceptance with a draft length of 7 tokens and a maximum output length of 4096 tokens. Acceptance length includes one additional target token, giving a maximum of 8.

Table notation:

• T: target model temperature.
• High / Low / No thinking: target model thinking modes.
• Prob. / Greedy: probabilistic sampling / argmax draft decoding.

Dataset	T=1, High, Prob.	T=1, Low, Prob.	T=1, No thinking, Prob.	T=0, High, Greedy
AIME25	4.1833	4.0749	4.8251	4.6493
GSM8K	4.4056	4.3266	4.9514	4.4845
MATH500	4.5942	4.6721	5.1758	5.0335
HumanEval	3.8743	4.1370	4.5859	4.4330
MBPP	3.8671	4.0179	4.3935	4.5525
Arena-Hard-v2	3.2172	3.1772	3.0499	4.1601
Average	4.0236	4.0676	4.4969	4.5522

On-Device Inference

We compare inference throughput with and without DSpark on NVIDIA RTX 4090 (llama.cpp) and Apple M5 Pro (MLX), using BF16 and quantized target models (Q4_K_M on RTX 4090 and INT4 on M5 Pro).

Dataset	BF16 · RTX 4090	BF16 · M5 Pro	Q4_K_M · RTX 4090	INT4 · M5 Pro
AIME25	2.22× 95 → 210 tok/s	2.62× 31 → 80 tok/s	2.00× 226 → 451 tok/s	1.69× 71 → 119 tok/s
GSM8K	2.49× 95 → 236 tok/s	2.57× 31 → 79 tok/s	2.35× 226 → 532 tok/s	1.66× 71 → 118 tok/s
MATH500	2.51× 95 → 239 tok/s	2.94× 31 → 90 tok/s	2.33× 226 → 527 tok/s	1.89× 71 → 135 tok/s
HumanEval	2.06× 95 → 196 tok/s	2.41× 31 → 74 tok/s	1.72× 226 → 389 tok/s	1.56× 71 → 110 tok/s
MBPP	1.88× 95 → 178 tok/s	2.49× 31 → 77 tok/s	1.85× 227 → 420 tok/s	1.61× 71 → 115 tok/s
Arena-Hard-v2	1.70× 94 → 160 tok/s	2.00× 31 → 61 tok/s	1.62× 225 → 365 tok/s	1.30× 71 → 92 tok/s
Average	2.14× 95 → 203 tok/s	2.50× 31 → 77 tok/s	1.98× 226 → 447 tok/s	1.62× 71 → 115 tok/s

Usage with llama.cpp

Use XHToken/llama.cpp to run Spark DSpark and send requests through an OpenAI-compatible API. The following commands are for Linux, Bash, and NVIDIA GPUs.

Branch: master (22c0ae1871b60123c5afc008a78fe9c6fffa2858). Integration into upstream llama.cpp is in progress.

1. Build

Clone the source and build:

git clone --branch master https://github.com/XHToken/llama.cpp.git
cd llama.cpp
cmake -S . -B build-cuda -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build-cuda --target llama-server -j8

2. Start the Server

set -o pipefail
./build-cuda/bin/llama-server \
  -m models/Spark-target.gguf \
  -md models/Spark-draft.gguf \
  --alias spark-dspark \
  --spec-type draft-dspark \
  --spec-draft-sampling greedy \
  --spec-draft-n-max 7 \
  --spec-draft-p-min 0 \
  -ngl 999 -ngld 999 \
  -ctk f16 -ctv f16 \
  -ctkd f16 -ctvd f16 \
  -fa on \
  -c 8192 -np 1 \
  -b 2048 -ub 512 -t 8 \
  --jinja \
  --host 127.0.0.1 --port 8080 \
  2>&1 | tee dspark-server.log

3. Send an Example Request

curl -fsS http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "spark-dspark",
    "messages": [
      {
        "role": "user",
        "content": "你好"
      }
    ],
    "temperature": 1.0,
    "top_p": 0.95,
    "max_tokens": 512,
    "stream": false
  }'

License

The Spark-X2.6 model series is licensed under the Apache 2.0 License.

Citation

If you find our work helpful, feel free to give us a cite.

@misc{sparkx2.6,
    title  = {Spark-X2.6-4B&1.7B: Advancing the Frontier of On-Device Agents for Real-World Tasks},
    author = {SparkLLM Team},
    year   = {2026}
}