inkling-2-mini-concise

Inkling 2 Mini that overthinks less, and scores higher on reasoning and code as a result.

The upstream model card warns that "the model sometimes became stuck in long reasoning loops, or repeatedly re-checked its own work without making progress." This checkpoint addresses that with a rank-one, gradient-free weight edit (weight-space programming). There is no training and no fine-tuning data: a direction is read from the model's own activations and compiled into six weight matrices.

Results

Setup: vLLM, thinking on (default reasoning_effort 0.9), temperature 0.9, max 32k tokens, one H200. The base is the unmodified thinkingmachines/Inkling-2-Mini. Δ is paired by problem, with a 95% bootstrap CI.

Benchmark Base concise Δ accuracy [95% CI] Δ thinking tokens [95% CI] Truncated at 32k
GPQA Diamond (198 × 2) 80.8 83.3 +2.5 [0.0, +5.3] −10.4% [−15.4, −5.6] 42 → 37
LiveCodeBench v6 (175, 2025-01…04) 65.1 68.6 +3.4 [−2.3, +9.1] −9.3% [−14.6, −3.7] 53 → 43
IFBench (300, loose) 78.3 73.7 −4.7 [−9.0, 0.0] −30.6% [−37.6, −23.3] 4 → 2
13 held-out coding/math/science tasks × 3 35/39 34/39 −2.6 [−12.8, +7.7] −21.4% 0 → 0

Mean thinking tokens, base → concise: GPQA 11,207 → 10,039; LCB 14,775 → 13,403; IFBench 4,998 → 3,467; held-out tasks 8,093 → 6,359.

Most of the gain comes from fewer runaway generations. Fewer traces hit the 32k limit, so questions the base model never finished now get answered. The cost is a borderline drop on strict instruction following (IFBench), where the edit shortens thinking the most.

Dose response (same direction, larger α). Stronger edits cut much deeper on hard tasks than on easy ones:

α GPQA LCB v6 IFBench thinking tokens (GPQA / LCB / IFBench)
0.2 (this model) +2.5 +3.4 −4.7 −10% / −9% / −31%
0.5 +1.5 −6.9 −15.3 −42% / −63% / −63%
0.7 −16.7 −29.1 −33.0 −82% / −93% / −85%

Do not scale the edit up.

What was changed

Edited tensors model.llm.layers.{18..23}.attn.wo_ud.weight (attention output projection of the six coda layers, after the loop)
Precision BF16, the same as in the original release. No re-quantization.
Edit W ← W + α · n · d″ uᵀ / ‖u‖², α = 0.2
Relative size ‖ΔW‖/‖W‖ = 0.28–0.58% per layer
Everything else byte-identical to thinkingmachines/Inkling-2-Mini (NVFP4/MXFP8 experts, DSpark speculator, vision tower)
  • d is the "about to finish thinking" direction: the mean of the block-residual partial state over the last 48 thinking tokens, minus the rest of the thinking. It was measured on 283 MATH-train (level 5) traces of this model.
  • Its component along an instruction-following direction (IFEval thinking minus code thinking, measured on prompts disjoint from IFBench) is removed. d″ corrects for the attention short-conv (attn_sconv, per-channel tap sums). u is the mean wo_ud input on thinking tokens, and n is the mean wo_ud output norm.
  • The coda layers run once per token. The looped core (layers 6–17, three passes) is not touched.
  • α was screened on MATH-train and confirmed on the benchmarks above. The held-out tasks were never used for selection.

Files in edit/: direction tensors (directions.npz), the in-process vLLM capture hook, and the scripts that build the edit.

Usage

Load it exactly like the original, with the official image vllm/vllm-openai:inkling-2-mini (Blackwell, DGX Spark):

vllm serve autotrust/inkling-2-mini-concise --reasoning-parser inkling

On Hopper (H100/H200), that image (commit 7cb5dae) ships MoE kernels built for SM100 only. We ran it with _moe_C_stable_libtorch.abi3.so taken from the vllm/vllm-openai:nightly-x86_64 image (2026-11-21, built for SM90). We also routed ops/conv_mix.py to its existing SM12x fallback path for SM90. On one H200, aggregate decode was about 3.6–5.3k tok/s at 100–256 concurrent requests.

License

Apache-2.0, inherited from thinkingmachines/Inkling-2-Mini. Use is also subject to the Thinking Machines Model Acceptable Use Policy. Modifications: the six attn.wo_ud.weight tensors listed above were replaced. The weights are stored as model-wsp-edit.safetensors plus model-stripped.safetensors, which holds everything else unchanged.

Method after weight-space programming (Takehi et al., 2026) and the norm-matched replication by Concept Country (2026).

Downloads last month
-
Safetensors
Model size
26B params
Tensor type
F32
·
BF16
·
F8_E4M3
·
U8
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for autotrust/inkling-2-mini-concise

Quantized
(142)
this model