Instructions to use autotrust/inkling-2-mini-concise with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use autotrust/inkling-2-mini-concise with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="autotrust/inkling-2-mini-concise") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("autotrust/inkling-2-mini-concise") model = AutoModelForMultimodalLM.from_pretrained("autotrust/inkling-2-mini-concise", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use autotrust/inkling-2-mini-concise with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "autotrust/inkling-2-mini-concise" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "autotrust/inkling-2-mini-concise", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/autotrust/inkling-2-mini-concise
- SGLang
How to use autotrust/inkling-2-mini-concise with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "autotrust/inkling-2-mini-concise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "autotrust/inkling-2-mini-concise", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "autotrust/inkling-2-mini-concise" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "autotrust/inkling-2-mini-concise", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use autotrust/inkling-2-mini-concise with Docker Model Runner:
docker model run hf.co/autotrust/inkling-2-mini-concise
inkling-2-mini-concise
Inkling 2 Mini that overthinks less, and scores higher on reasoning and code as a result.
The upstream model card warns that "the model sometimes became stuck in long reasoning loops, or repeatedly re-checked its own work without making progress." This checkpoint addresses that with a rank-one, gradient-free weight edit (weight-space programming). There is no training and no fine-tuning data: a direction is read from the model's own activations and compiled into six weight matrices.
Results
Setup: vLLM, thinking on (default reasoning_effort 0.9), temperature 0.9, max 32k tokens, one H200. The base is the
unmodified thinkingmachines/Inkling-2-Mini. Δ is paired by problem, with a 95% bootstrap CI.
| Benchmark | Base | concise | Δ accuracy [95% CI] | Δ thinking tokens [95% CI] | Truncated at 32k |
|---|---|---|---|---|---|
| GPQA Diamond (198 × 2) | 80.8 | 83.3 | +2.5 [0.0, +5.3] | −10.4% [−15.4, −5.6] | 42 → 37 |
| LiveCodeBench v6 (175, 2025-01…04) | 65.1 | 68.6 | +3.4 [−2.3, +9.1] | −9.3% [−14.6, −3.7] | 53 → 43 |
| IFBench (300, loose) | 78.3 | 73.7 | −4.7 [−9.0, 0.0] | −30.6% [−37.6, −23.3] | 4 → 2 |
| 13 held-out coding/math/science tasks × 3 | 35/39 | 34/39 | −2.6 [−12.8, +7.7] | −21.4% | 0 → 0 |
Mean thinking tokens, base → concise: GPQA 11,207 → 10,039; LCB 14,775 → 13,403; IFBench 4,998 → 3,467; held-out tasks 8,093 → 6,359.
Most of the gain comes from fewer runaway generations. Fewer traces hit the 32k limit, so questions the base model never finished now get answered. The cost is a borderline drop on strict instruction following (IFBench), where the edit shortens thinking the most.
Dose response (same direction, larger α). Stronger edits cut much deeper on hard tasks than on easy ones:
| α | GPQA | LCB v6 | IFBench | thinking tokens (GPQA / LCB / IFBench) |
|---|---|---|---|---|
| 0.2 (this model) | +2.5 | +3.4 | −4.7 | −10% / −9% / −31% |
| 0.5 | +1.5 | −6.9 | −15.3 | −42% / −63% / −63% |
| 0.7 | −16.7 | −29.1 | −33.0 | −82% / −93% / −85% |
Do not scale the edit up.
What was changed
| Edited tensors | model.llm.layers.{18..23}.attn.wo_ud.weight (attention output projection of the six coda layers, after the loop) |
| Precision | BF16, the same as in the original release. No re-quantization. |
| Edit | W ← W + α · n · d″ uᵀ / ‖u‖², α = 0.2 |
| Relative size | ‖ΔW‖/‖W‖ = 0.28–0.58% per layer |
| Everything else | byte-identical to thinkingmachines/Inkling-2-Mini (NVFP4/MXFP8 experts, DSpark speculator, vision tower) |
dis the "about to finish thinking" direction: the mean of the block-residual partial state over the last 48 thinking tokens, minus the rest of the thinking. It was measured on 283 MATH-train (level 5) traces of this model.- Its component along an instruction-following direction (IFEval thinking minus code thinking, measured on prompts
disjoint from IFBench) is removed.
d″corrects for the attention short-conv (attn_sconv, per-channel tap sums).uis the meanwo_udinput on thinking tokens, andnis the meanwo_udoutput norm. - The coda layers run once per token. The looped core (layers 6–17, three passes) is not touched.
- α was screened on MATH-train and confirmed on the benchmarks above. The held-out tasks were never used for selection.
Files in edit/: direction tensors (directions.npz), the in-process vLLM capture hook, and the scripts
that build the edit.
Usage
Load it exactly like the original, with the official image vllm/vllm-openai:inkling-2-mini (Blackwell, DGX Spark):
vllm serve autotrust/inkling-2-mini-concise --reasoning-parser inkling
On Hopper (H100/H200), that image (commit 7cb5dae) ships MoE kernels built for SM100 only. We ran it with
_moe_C_stable_libtorch.abi3.so taken from the vllm/vllm-openai:nightly-x86_64 image (2026-11-21, built for
SM90). We also routed ops/conv_mix.py to its existing SM12x fallback path for SM90. On one H200, aggregate decode
was about 3.6–5.3k tok/s at 100–256 concurrent requests.
License
Apache-2.0, inherited from thinkingmachines/Inkling-2-Mini. Use is also subject to the Thinking Machines
Model Acceptable Use Policy. Modifications: the six
attn.wo_ud.weight tensors listed above were replaced. The weights are stored as model-wsp-edit.safetensors
plus model-stripped.safetensors, which holds everything else unchanged.
Method after weight-space programming (Takehi et al., 2026) and the norm-matched replication by Concept Country (2026).
- Downloads last month
- -
Model tree for autotrust/inkling-2-mini-concise
Base model
thinkingmachines/Inkling-2-Mini