Instructions to use luisbrel/weight-skills with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use luisbrel/weight-skills with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("luisbrel/weight-skills") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- MLX LM
How to use luisbrel/weight-skills with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "luisbrel/weight-skills" --prompt "Once upon a time"
- Atomic Chat
Weight Skills: compiled context for local agents
Paper (latest version): Zero-Skills in Context β DOI 10.5281/zenodo.24122803 Β·
Measurement addendum: MEASUREMENT_ADDENDUM.md
π¬ Solo research project. Built and measured as an independent research effort. Please leave a comment, poke holes in it, and share your findings.
β‘ What this is
Agent frameworks paste the same long instructions into every request: a persona, a skills manual, a framework preamble, a style guide. That text is re-read (prefilled) on every single turn.
There are two closed-form ways to stop paying for it. This repository contains both, their artefacts and the measurements:
| Compiled prompt cache | Cartridge (weight delta) | |
|---|---|---|
| What it does | Reuses the prefill of the real prompt text | Removes the prompt text and adds a low-rank FFN delta |
| Effect on behaviour | None β bit-identical to cold | Changes the function (approximate by construction) |
| Prompt tokens | 0 for the cached prefix | 0 |
| Use it for | Personas, skills manuals, frameworks, SOPs | Format/style habits and prompt-cost elimination |
Rule of thumb: compile personas and skills as prompt caches when behaviour must be exactly what the prompt specifies; use cartridges when an approximation is acceptable and the prompt cost must go.
β Measured results
1. Compiled prompt caches are exact under the aligned contract
Compile the prompt up to the cold path's prefill-chunk boundary, re-prefill the remainder with the question, and serve only the suffix; a chunk mismatch between compile and serve is refused with a reason.
| Setting (M4 Max 128 GB, MLX) | Tokens | Identity vs cold | Max abs logit Ξ | Speedup |
|---|---|---|---|---|
| occamy 35B MoE / heretic 35B / nemotron 30B / proto 30B | 2k / 8k / 32k (20/20 cells) | bit-identical per model | 0.0 | median TTFT 9.8β11.8Γ |
| occamy 35B MoE | 65,536 | bit-identical round-trip | 0.0 | 195.7Γ (0.356 s vs 69.76 s) |
| heretic 35B / occamy, 8k serving benchmark | 8,192 | 10/10 identity | 0.0 | 45.4Γ / 35.2Γ |
| qwen2-0.5B, 8-bit KV cache | 2k / 8k | all served rows | 0.0 | 20.1Γ / 35.3Γ |
| chunk edges (n = kC Β± 1) | 2,047β4,097 | 24/24 bit-identical | 0.0 | β |
Multi-tenant serving (25 runnable cells: 8/12/16/24/32 tenants Γ 2k/4k/8k prefixes Γ 1024/2048 chunks, reports/MULTITENANT_ALIGNED_SCALE.json @ b3ec2b6, M4 Max 128GB MLX):
100 % identity (1000/1000 served rows, Ξlogit 0.0), telemetry exact, forced chunk mismatch refused
every block. Warm speedups are 1.59β5.94Γ across the grid; at 32 tenants they are 1.001β1.009Γ because
the first serve of each cache pays a materialisation cost that scales with the prefix, so the benefit
tracks cache reuse rather than tenant count.
The aligned contract is what makes a compiled cache exact (reports/MULTITENANT_ALIGNED_SCALE.json @ b3ec2b6, reports/TASK_FAMILY_COMPARISON.json @ 03af0f6, M4 Max 128GB MLX): plain compiled serving diverges on 64β85% (64.0% heretic, 72.7% nemotron, 84.9% occamy) of SWE-style cells and on 5.0% of long-context cells, while the aligned control diverges 0/33. For canonical layer-7 qwen2-0.5B weight edits, $C_0$-whitening + subject-last-token positioning (W3, DenseSkillCompiler @ c3c4f71) reaches 15/24 (62.5%) bf16 (reports/WHITENED_PARAPHRASE_TRANSFER.json @ b1fee0d) and 14/24 (58.3%) 8-bit affine (group_size=64) held-out paraphrase transfer with 72/72 active neighbours (128/132 = 96.97% 8-bit specificity, 0/132 off-diag; G.140) under leak-free layer-7 activation-key cosine routing (G.141: 23/24 = 95.8% held-out route, 24/24 argmax, 72/72 neighbour pair non-interference, reports/WHITENED_8BIT_AND_ROUTER.json @ 0b118c3). Use the
aligned mode when outputs must match cold execution.
2. Cartridges: prompt cost eliminated, decode unchanged
Six checkpoints, Apple Silicon M4 Max, 4,000-token system prompt vs a compiled cartridge:
| Model | Prefill tax (B Γ· A) | Decode retained | Mount | Restoration |
|---|---|---|---|---|
| Gemma 4 4B (dense) | 9.37Γ | 0.96Γ | 1.93 ms | exact |
| ThinkLite 27B (dense) | 24.84Γ | 0.96Γ | 2.83 ms | exact |
| Occamy 35B MoE | 6.13Γ | 0.98Γ | 2.23 ms | exact |
| Proto-Nemotron 30B MoE | 16.09Γ | 1.00Γ | 1.29 ms | exact |
| Nemotron 3.5 Lightning 30B | 14.26Γ | 1.03Γ | 1.16 ms | exact |
| Jimmy-4B (Gemma 4 4B derivative) | 8.27Γ | 0.96Γ | 1.69 ms | exact |
7/7 restoration checks per model and 100/100 bit-identical mount cycles. The benefit is eliminated prefill and KV growth; per-token decode throughput is unchanged (0.96β1.03Γ).
3. Point edits, 8-bit push-through, and activation-key routing (G.117βG.141)
Construction (qwen2-0.5B, layer 7, 12 facts) |
Direct | Dev PS (36) |
Held-out Eval PS (24) |
Reading |
|---|---|---|---|---|
| 12-fact summed ROME / MEMIT / sequential refit | 0/12 / 2/12 / 1/12 |
n/a |
0/60 (all) |
one shared layer holds one fact (G.112/G.123) |
Unwhitened -1 ROME / 8-bit ROME push-through |
11/12 / 11/12 |
n/a |
0/60 (all) |
8-bit carries single-fact; -1 keys prompt-brittle (G.126) |
| 4-bit push-through of single-fact ROME ΞW | 0/12 |
n/a |
0/60 (all) |
ΞW below 9.657% 4-bit quantisation floor (G.122) |
$C_0$-whitened subj_last_idx W3 (bf16, oracle dispatch) |
12/12 |
36/36 |
15/24 (62.5%) |
72/72 neighbours, 0/132 off-diag (WHITENED_PARAPHRASE_TRANSFER.json @ b1fee0d) |
$C_0$-whitened subj_last_idx W3 (8-bit path a, oracle) |
12/12 |
36/36 |
14/24 (58.3%) |
10.2β12.96Γ above 0.7718% 8-bit floor; 72/72 neighbours (G.140) |
Activation-key cosine router (bf16 / 8-bit, oracle-free) |
12/12 |
36/36 |
15/24 (bf16) / 14/24 (8-bit) |
23/24 (95.8%) route (24/24 argmax); 5/5 NS + 6/6 neigh abstain (G.141, WHITENED_8BIT_AND_ROUTER.json @ 0b118c3) |
K=24 scaling (bf16 / 8-bit, qwen2-0.5b, G.143) |
24/24 |
72/72 (bf16) / 71/72 (8-bit) |
32/48 (31/48 routed) / 28/48 |
144/144 neighbours, 46/48 cosine route (48/48 argmax, MULTIBILLION_WSSC_SCALING.json @ 6156959) |
LFM2.5-2.6B-8bit layer-13 w2 (G.143 PARTIAL) |
12/12 |
36/36 |
8/24 (3/24 routed) |
43/72 neighbours, 0 restore failures, kl_anchor_max=0.1185 (MULTIBILLION_WSSC_SCALING.json @ 6156959) |
Prefix-marginalized W3 (bf16 / 8-bit, G.146 mixed) |
11/12 |
30/36 |
11/24 (14/24 norm) / 9/24 |
Prefixed 9/12 -> 10/12 direct & 10 eval; 2-proto router 23/24 (PREFIX_MARGINALIZED_WSSC.json @ 6156959) |
4. Two-regime hybrid agent runtime & safety (G.145, TWO_REGIME_HYBRID_RUNTIME.json @ 6156959)
TwoRegimeAgentRuntime (two_regime_runtime.py) passes 4/4 verification gates in an interleaved MLX session (26/26 CPU unit tests):
- Regime 1 (Aligned chunked KV,
CHUNK_SIZE=64,cut=64):12/12compliant procedural JSON,12/12aligned exact token match vs cold, SHA-256 tamper rejection, exact rollback trim. - Regime 2 (Routed
8-bitW3,tau=0.60):12/12direct,23/24dev-24,35/36dev-36,15/24held-out eval,0restore failures (71apply/restore cycles). - Regime 3 (Deterministic external
PolicyGate):24/24adversarial blocked,8/8benign +10/10defensive allowed (0false positives), credential leak intercepted (interleaved_session.ok=True).
π¦ Authentic Cartridge Release Inventory (11 weight-skills/cartridge@1 Files)
All 11 published .safetensors files under cartridges/ conform to schema weight-skills/cartridge@1 (manifest.json records exact byte sizes, SHA-256 digests, ranks, and target layers):
| Checkpoint Directory | Cartridge File | Skill / Role | Rank / Layers | Status Note |
|---|---|---|---|---|
base_nemotron_30b/ |
base_nemotron_30b.aws-cso-r16.safetensors |
aws-cso procedural skill |
r=16, [24, 31, 38] |
Verified mount/restore |
gemma4_4b/ |
clu-completion-r32.safetensors |
clu-completion persona |
r=32, [16, 20, 24, 28] |
Verified mount/restore |
gemma4_4b/ |
gemma4_4b.aws-cso-r16.safetensors |
aws-cso procedural skill |
r=16, [19, 24, 29] |
Verified mount/restore |
jimmy_4b/ |
jimmy_4b.aws-cso-r16.safetensors |
aws-cso procedural skill |
r=16, [19, 24, 29] |
Verified mount/restore |
occamy_35b_moe/ |
occamy_35b_moe.aws-cso-r16-vlm.safetensors |
aws-cso (mlx-vlm text tower) |
r=16, [18, 23, 28] |
Verified mount/restore |
occamy_35b_moe/ |
occamy_35b_moe.aws-cso-r16.safetensors |
aws-cso procedural skill |
r=16, [18, 23, 28] |
Verified mount/restore |
occamy_35b_moe/ |
occamy_35b_moe.aws-cso-r4-vlm.safetensors |
aws-cso (mlx-vlm, n_probes=4) |
r=4, [18, 23, 28] |
Compact rank-4 check |
proto_nemotron_30b/ |
proto_nemotron_30b.aws-cso-r16.safetensors |
aws-cso procedural skill |
r=16, [24, 31, 38] |
Verified mount/restore |
thinklite_27b/ |
thinklite_27b.aws-cso-r16.safetensors |
aws-cso procedural skill |
r=16, [29, 37, 45] |
Verified mount/restore |
refusal/ |
qwen36-35b-heretic-4bit.guardrail-r16.safetensors |
guardrail-strict-refusal (r=16) |
r=16, [18, 23, 28] |
Experimental negative (0/24 standalone refusal; use PolicyGate) |
refusal/ |
qwen36-35b-heretic-4bit.guardrail-r64-L8-P8.safetensors |
guardrail-strict-refusal (r=64, 8L) |
r=64, 8 layers [18..28] |
Experimental negative (0/24 standalone refusal; use PolicyGate) |
Note: Obsolete ungrounded files from earlier draft uploads were purged in favour of this byte-verified 11-cartridge
weight-skills/cartridge@1inventory andtwo_regime_runtime.py.
π Quickstart
Hybrid two-regime runtime (aligned KV prefix + cosine-routed 8-bit W3 + external PolicyGate):
# See two_regime_runtime.py (G.145, 26/26 unit tests + 4/4 live MLX gates PASS)
from bender_bench.cartridges.two_regime_runtime import TwoRegimeAgentRuntime, PolicyGate
gate = PolicyGate() # blocks 24/24 adversarial requests, 0 false positives
# Align prefix at CHUNK_SIZE=64 with SHA-256 verification & cold chunked tail parity,
# and dispatch routed 8-bit W3 adapters at cosine threshold tau=0.60 with exact restore.
Mounting a Dynamic Cartridge (zero-prompt approximate injection, exact unmount):
import mlx_lm
from safetensors import safe_open
# Inspect metadata & load cartridge tensors from cartridges/<model>/<file>.safetensors
with safe_open("cartridges/proto_nemotron_30b/proto_nemotron_30b.aws-cso-r16.safetensors", framework="numpy") as f:
print(f.metadata()["weight_skills"]) # schema: weight-skills/cartridge@1
The research runtime & tests (two_regime_runtime.py, ParetoAnchorGate.strict_rsi_gate(min_samples_per_suite=50)) ship alongside weight_space_skill_compilation_paper.pdf and MEASUREMENT_ADDENDUM.md.
π§ͺ Tests and scope
- 26/26 two-regime runtime unit tests (
tests/test_two_regime_runtime.py), 25/25 powered Pareto gate unit tests (tests/test_pareto_gate.py), and 250 library tests (weight-skillspackage); cross-model Batch 11 audit passed 99/99 checks (reports/LOOP_BATCH11_AUDIT.json@05d4530). - Benchmarks measured on Apple M4 Max (128 GB) with MLX, greedy decoding, on committed artefacts; the full protocol and provenance are in
weight_space_skill_compilation_paper.pdfandMEASUREMENT_ADDENDUM.md. - Weight deltas change a function; they are not a standalone security boundary (
0/24refusal installed when mounted inGUARDRAIL_MECHANISM_AUDIT.json). For exact behaviour use an aligned compiled prompt cache; for safety enforcement use the external deterministicPolicyGate(24/24blocked,0false positives inG.145). Benchmark promotion gates enforcen >= 50samples per suite (G.142,ParetoAnchorGate.strict_rsi_gate(min_samples_per_suite=50)inreports/PARETO_POWERED_GATE_AUDIT.json@353dfb8; empirically rejecting the underpoweredn=56Gemma-4-4B pilot inG.144-4.0%MATH-500 ->REJECTED_REGRESSION).
Citation
@article{machado2026zeroskills,
title = {Zero-Skills in Context: Sub-Second Algebraic Compilation of Agent Procedures into Feed-Forward Weights},
author = {Machado, Luis},
year = {2026},
doi = {10.5281/zenodo.24122803},
url = {https://doi.org/10.5281/zenodo.24122803},
note = {Concept DOI β resolves to the latest version}
}
Quantized