Instructions to use AtomicChat/Spark-X2.6-1.7B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AtomicChat/Spark-X2.6-1.7B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use AtomicChat/Spark-X2.6-1.7B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AtomicChat/Spark-X2.6-1.7B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtomicChat/Spark-X2.6-1.7B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Ollama
How to use AtomicChat/Spark-X2.6-1.7B-GGUF with Ollama:
```
ollama run hf.co/AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use AtomicChat/Spark-X2.6-1.7B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use AtomicChat/Spark-X2.6-1.7B-GGUF with Docker Model Runner:
```
docker model run hf.co/AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M
```

Lemonade

How to use AtomicChat/Spark-X2.6-1.7B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Spark-X2.6-1.7B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use AtomicChat/Spark-X2.6-1.7B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AtomicChat/Spark-X2.6-1.7B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AtomicChat/Spark-X2.6-1.7B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

How to Run Spark-X2.6 1.7B Locally

GGUF builds of XHToken's Spark-X2.6 1.7B, made from the original BF16 weights with our own importance matrix, and measured against every other build we could find. The calibration corpora behind our builds are public.

You can now run Spark-X2.6 in Atomic Chat with thinking on by default (llama.cpp engine b11946 or newer).
AD-Q4_K_M, 1.17 GB on disk, matches the original's token choice 88.0% of the time; the official Q4_K_M (1.77 GB) 84.8%.
Every other GGUF build we could find is measured on the same stand below, including where it beats ours.

Every Spark-X2.6 1.7B GGUF we could find, ours and others', measured against the BF16 original on the same held-out text. The dashed line is the best file from anyone but us at each size; red is where it is closer to the original than ours.

Pick a file

Every number is measured on one stand; how is at the bottom, and the raw logs are in the metrics repo. KL divergence is how far a file's predictions drift from the original weights (lower is better, zero is identical); top-1 is how often it picks the same next token as the original.

File	Size on disk	KL divergence	top-1
`Spark-X2.6-1.7B-Q8_0.gguf`	1.82 GB	0.00236	97.58%	closest to the original
`Spark-X2.6-1.7B-AD-Q6_K.gguf`	1.61 GB	0.00534	96.20%	near Q8_0 for 0.2 GB less
`Spark-X2.6-1.7B-AD-Q5_K_M.gguf`	1.31 GB	0.02173	92.51%	28% smaller than Q8_0
`Spark-X2.6-1.7B-AD-Q4_K_M.gguf`	1.17 GB	0.05602	87.99%	recommended
`Spark-X2.6-1.7B-AD-IQ4_XS.gguf`	1.11 GB	0.07414	86.35%	the size of llama.cpp's Q4_K_M, with 42% lower KL divergence
`Spark-X2.6-1.7B-AD-IQ4_XS-IQ3_S.gguf`	0.96 GB	0.16981	79.58%	smallest we publish

AD- marks an Atomic Dynamic layout: each tensor group gets its own type, set by measurement, and the name says what the feed-forward tensors got. llama.cpp's default Q4_K_M of the same BF16 (1.11 GB, KL divergence 0.1288, top-1 82.13%) is the baseline every AD- file beats at its size; it is not published here. AD-IQ3_S (0.91 GB, 76.3% top-1) saves 55 MB over AD-IQ4_XS-IQ3_S for a large drop, and AD-IQ3_XXS (0.83 GB) fell under our 76% top-1 bar, so neither is published.

How these compare to other builds

Their files, downloaded and measured with our reference, text, context and llama.cpp build. Their own published numbers are never used. Each one is set against our best published file at its size or smaller (2% slack for files that are the same size give or take a few megabytes). The chart draws a line between our files; this table only compares files you can download, so a file that falls between two of ours can be closer to the original than the smaller of them.

Their file	Size	KL divergence	top-1	Our best at that size	Size	KL divergence	top-1	Closer to the original
official `Q8_0`	1.82 GB	0.0027	97.50%	`Q8_0`	1.82 GB	0.0024	97.58%	ours
official `Q4_K_M`	1.77 GB	0.0942	84.82%	`AD-Q6_K`	1.61 GB	0.0053	96.20%	ours
mradermacher `i1-Q6_K`	1.41 GB	0.0143	94.14%	`AD-Q5_K_M`	1.31 GB	0.0217	92.51%	theirs
mradermacher `i1-Q5_K_M`	1.25 GB	0.0376	90.22%	`AD-Q4_K_M`	1.17 GB	0.0560	87.99%	theirs
mradermacher `i1-Q4_K_M`	1.11 GB	0.1360	82.08%	`AD-IQ4_XS`	1.11 GB	0.0741	86.35%	ours
llama.cpp default `Q4_K_M`	1.11 GB	0.1288	82.13%	`AD-IQ4_XS`	1.11 GB	0.0741	86.35%	ours
mradermacher `i1-IQ4_XS`	1.02 GB	0.1531	80.92%	`AD-IQ4_XS-IQ3_S`	0.96 GB	0.1698	79.58%	theirs
mradermacher `i1-IQ3_M`	0.89 GB	0.4186	69.43%	we publish none this small				no file of ours

Which one fits your card

The file has to fit, and so does the context. With llama.cpp's full-size cache this model keeps 56 KB per token (0.4 GB at 8k context, 1.8 GB at 32k); three of every four blocks attend over a 512-token window only, so the default sliding-window cache needs less. The table plans for the full-size cache and leaves 1 GB for compute buffers.

Your card	File	Leaves room for
3 GB	`AD-Q6_K`	about 9k context
4 GB	`Q8_0`	about 23k context
6 GB	`Q8_0`	about 60k context
8 GB	`Q8_0`	about 97k context

Running it

hf download AtomicChat/Spark-X2.6-1.7B-GGUF Spark-X2.6-1.7B-AD-Q4_K_M.gguf --local-dir .
llama-server -m Spark-X2.6-1.7B-AD-Q4_K_M.gguf -ngl 99 -c 16384 --jinja

The chat template is inside every file. Thinking is on by default; per request, turn it off or make it shorter (the template takes high, the default, and low only):

{"messages": [...], "chat_template_kwargs": {"enable_thinking": false}}
{"messages": [...], "chat_template_kwargs": {"reasoning_effort": "low"}}

Checked on llama-server (a34de8618) with AD-Q4_K_M: a chat answer with thinking, thinking off, low effort, and a tool call parsed into tool_calls.

Speed

AD-Q4_K_M on one NVIDIA A10 (24 GB), llama-bench, flash attention on, all layers on the GPU, llama.cpp a34de8618:

test	tokens per second
pp512	12,351
pp8192	12,015
tg128	266.2

What we found while building these

The official GGUF keeps the 131,072-row token embedding in BF16 and writes the tied head again as output.weight in Q8_0, so its Q4_K_M is 1.77 GB against llama.cpp's 1.11 GB for the same preset. Here the embedding is the head (tied), at the head's type.
The fused attn_qkv and the per-head sigmoid gate on the attention output (attn_gate) see the largest activations in the importance matrix, most of all in the full-attention blocks; they stay one step above the feed-forward tensors, and attn_gate (a few thousand weights per block) stays q8_0.
This model is sensitive to quantization: llama.cpp's own Q4_K_M keeps 82.1% top-1 here, where our AD-Q4_K_M of Qwen3.8-27B keeps 95.6%.
AD-IQ3_S (0.91 GB, 76.3% top-1) saves 55 MB over AD-IQ4_XS-IQ3_S for a large drop, and AD-IQ3_XXS (0.83 GB) fell under our 76% top-1 bar, so neither is published.

The calibration data

Our importance matrix comes from a corpus we built for this model and published: AtomicChat/calib-corpora, recipe spark-x26. It is 4,787,678 tokens across 2,986 documents:

Part	Share
agentic and tool use	25.2%
code	19.3%
reasoning	15.7%
multilingual	14.6%
long context	12.1%
vocabulary sweep	5.4%
structured data	4.6%
graphics	3.1%

Every conversation is rendered through this model's own chat template, so the chat markup appears as the single tokens the model actually sees.

If you build your own importance matrix from this corpus, pass --parse-special to llama-imatrix. Without it the chat markup is read as ordinary punctuation.

How we measured

Reference: the original BF16 weights, converted to GGUF and run unquantized. Perplexity on the held-out set is 15.3634 plus or minus 0.10987.

Held-out text: eval_neutral from our calibration dataset, never used for calibration. 90 chunks at 4096 context.

Metric: per-token KL divergence of each file's predictions against the reference, plus top-1 agreement.

Hardware: 1x A100-SXM4-40GB.

The reference logits (48 GB) are not published. To measure your own build at the same point, make them from the BF16 (convert_hf_to_gguf.py on the original) and the held-out text in the calibration dataset:

llama-perplexity -m Spark-X2.6-1.7B-BF16.gguf -f eval_neutral.txt \
  --kl-divergence-base base-neutral.kld -c 4096 -ngl 99
llama-perplexity -m your-quant.gguf -f eval_neutral.txt \
  --kl-divergence-base base-neutral.kld --kl-divergence -c 4096 -ngl 99

Reproducing a file

The importance matrix was collected on the BF16 weights rather than on a quantized copy, in one run:

llama-imatrix -m Spark-X2.6-1.7B-BF16.gguf -f calib_train.txt -o imatrix.gguf \
  -ngl 99 -c 512 -b 8192 -ub 8192 --parse-special

Then, for AD-Q4_K_M:

llama-quantize \
  --imatrix imatrix.gguf \
  --tensor-type '^blk\.(0|1|23|24|25|26|27)\.ffn_(down|gate|up)\.weight$=q5_k' \
  --tensor-type '^blk\.(2|3|4|5)\.ffn_(down|gate|up)\.weight$=q5_k' \
  --tensor-type '^blk\.\d+\.ffn_down\.weight$=q4_k' \
  --tensor-type '^blk\.\d+\.ffn_gate\.weight$=q4_k' \
  --tensor-type '^blk\.\d+\.ffn_up\.weight$=q4_k' \
  --tensor-type '^blk\.\d+\.attn_qkv\.weight$=q6_k' \
  --tensor-type '^blk\.\d+\.attn_gate\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.attn_output\.weight$=q6_k' \
  --tensor-type '^token_embd\.weight$=q6_k' \
  --override-kv general.file_type=int:15 \
  Spark-X2.6-1.7B-BF16.gguf Spark-X2.6-1.7B-AD-Q4_K_M.gguf Q8_0

The rules for every other file are in ladder/ in the metrics repo, next to its quantization log. The first rule that matches a tensor wins.

Model details

28 layers, hidden size 2048, feed-forward size 6656, vocabulary 131,072, context 1,048,576.

Needs a recent llama.cpp with spark2_5 architecture support. Built and tested with llama.cpp b12015 (a34de8618).

License

Apache-2.0, as the original: XHToken/Spark-X2.6-1.7B. The original's LICENSE file is in this repo. Quantized by AtomicChat.

Downloads last month: 495

GGUF

Model size

2B params

Architecture

spark2_5

Hardware compatibility

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for AtomicChat/Spark-X2.6-1.7B-GGUF

Base model

XHToken/Spark-X2.6-1.7B-Base

Finetuned

XHToken/Spark-X2.6-1.7B

Quantized

(13)

this model

Collection including AtomicChat/Spark-X2.6-1.7B-GGUF

Spark-X2.6

Collection

AtomicChat AD GGUF of XHToken's Spark-X2.6 4B and 1.7B, measured with every other build on one stand. • 4 items • Updated 2 days ago