Weight Skills β€” zero-token compiled context, bit-identical serving

Weight Skills: compiled context for local agents

Paper (latest version): Zero-Skills in Context β€” DOI 10.5281/zenodo.24122803 Β· Measurement addendum: MEASUREMENT_ADDENDUM.md

πŸ’¬ Solo research project. Built and measured as an independent research effort. Please leave a comment, poke holes in it, and share your findings.


⚑ What this is

Agent frameworks paste the same long instructions into every request: a persona, a skills manual, a framework preamble, a style guide. That text is re-read (prefilled) on every single turn.

There are two closed-form ways to stop paying for it. This repository contains both, their artefacts and the measurements:

Compiled prompt cache Cartridge (weight delta)
What it does Reuses the prefill of the real prompt text Removes the prompt text and adds a low-rank FFN delta
Effect on behaviour None β€” bit-identical to cold Changes the function (approximate by construction)
Prompt tokens 0 for the cached prefix 0
Use it for Personas, skills manuals, frameworks, SOPs Format/style habits and prompt-cost elimination

Rule of thumb: compile personas and skills as prompt caches when behaviour must be exactly what the prompt specifies; use cartridges when an approximation is acceptable and the prompt cost must go.


βœ… Measured results

1. Compiled prompt caches are exact under the aligned contract

Compile the prompt up to the cold path's prefill-chunk boundary, re-prefill the remainder with the question, and serve only the suffix; a chunk mismatch between compile and serve is refused with a reason.

Setting (M4 Max 128 GB, MLX) Tokens Identity vs cold Max abs logit Ξ” Speedup
occamy 35B MoE / heretic 35B / nemotron 30B / proto 30B 2k / 8k / 32k (20/20 cells) bit-identical per model 0.0 median TTFT 9.8–11.8Γ—
occamy 35B MoE 65,536 bit-identical round-trip 0.0 195.7Γ— (0.356 s vs 69.76 s)
heretic 35B / occamy, 8k serving benchmark 8,192 10/10 identity 0.0 45.4Γ— / 35.2Γ—
qwen2-0.5B, 8-bit KV cache 2k / 8k all served rows 0.0 20.1Γ— / 35.3Γ—
chunk edges (n = kC Β± 1) 2,047–4,097 24/24 bit-identical 0.0 β€”

Multi-tenant serving (25 runnable cells: 8/12/16/24/32 tenants Γ— 2k/4k/8k prefixes Γ— 1024/2048 chunks, reports/MULTITENANT_ALIGNED_SCALE.json @ b3ec2b6, M4 Max 128GB MLX): 100 % identity (1000/1000 served rows, Ξ”logit 0.0), telemetry exact, forced chunk mismatch refused every block. Warm speedups are 1.59–5.94Γ— across the grid; at 32 tenants they are 1.001–1.009Γ— because the first serve of each cache pays a materialisation cost that scales with the prefix, so the benefit tracks cache reuse rather than tenant count.

The aligned contract is what makes a compiled cache exact (reports/MULTITENANT_ALIGNED_SCALE.json @ b3ec2b6, reports/TASK_FAMILY_COMPARISON.json @ 03af0f6, M4 Max 128GB MLX): plain compiled serving diverges on 64–85% (64.0% heretic, 72.7% nemotron, 84.9% occamy) of SWE-style cells and on 5.0% of long-context cells, while the aligned control diverges 0/33. For canonical layer-7 qwen2-0.5B weight edits, $C_0$-whitening + subject-last-token positioning (W3, DenseSkillCompiler @ c3c4f71) reaches 15/24 (62.5%) bf16 (reports/WHITENED_PARAPHRASE_TRANSFER.json @ b1fee0d) and 14/24 (58.3%) 8-bit affine (group_size=64) held-out paraphrase transfer with 72/72 active neighbours (128/132 = 96.97% 8-bit specificity, 0/132 off-diag; G.140) under leak-free layer-7 activation-key cosine routing (G.141: 23/24 = 95.8% held-out route, 24/24 argmax, 72/72 neighbour pair non-interference, reports/WHITENED_8BIT_AND_ROUTER.json @ 0b118c3). Use the aligned mode when outputs must match cold execution.

2. Cartridges: prompt cost eliminated, decode unchanged

Six checkpoints, Apple Silicon M4 Max, 4,000-token system prompt vs a compiled cartridge:

Model Prefill tax (B Γ· A) Decode retained Mount Restoration
Gemma 4 4B (dense) 9.37Γ— 0.96Γ— 1.93 ms exact
ThinkLite 27B (dense) 24.84Γ— 0.96Γ— 2.83 ms exact
Occamy 35B MoE 6.13Γ— 0.98Γ— 2.23 ms exact
Proto-Nemotron 30B MoE 16.09Γ— 1.00Γ— 1.29 ms exact
Nemotron 3.5 Lightning 30B 14.26Γ— 1.03Γ— 1.16 ms exact
Jimmy-4B (Gemma 4 4B derivative) 8.27Γ— 0.96Γ— 1.69 ms exact

7/7 restoration checks per model and 100/100 bit-identical mount cycles. The benefit is eliminated prefill and KV growth; per-token decode throughput is unchanged (0.96–1.03Γ—).

3. Point edits, 8-bit push-through, and activation-key routing (G.117–G.141)

Construction (qwen2-0.5B, layer 7, 12 facts) Direct Dev PS (36) Held-out Eval PS (24) Reading
12-fact summed ROME / MEMIT / sequential refit 0/12 / 2/12 / 1/12 n/a 0/60 (all) one shared layer holds one fact (G.112/G.123)
Unwhitened -1 ROME / 8-bit ROME push-through 11/12 / 11/12 n/a 0/60 (all) 8-bit carries single-fact; -1 keys prompt-brittle (G.126)
4-bit push-through of single-fact ROME Ξ”W 0/12 n/a 0/60 (all) Ξ”W below 9.657% 4-bit quantisation floor (G.122)
$C_0$-whitened subj_last_idx W3 (bf16, oracle dispatch) 12/12 36/36 15/24 (62.5%) 72/72 neighbours, 0/132 off-diag (WHITENED_PARAPHRASE_TRANSFER.json @ b1fee0d)
$C_0$-whitened subj_last_idx W3 (8-bit path a, oracle) 12/12 36/36 14/24 (58.3%) 10.2–12.96Γ— above 0.7718% 8-bit floor; 72/72 neighbours (G.140)
Activation-key cosine router (bf16 / 8-bit, oracle-free) 12/12 36/36 15/24 (bf16) / 14/24 (8-bit) 23/24 (95.8%) route (24/24 argmax); 5/5 NS + 6/6 neigh abstain (G.141, WHITENED_8BIT_AND_ROUTER.json @ 0b118c3)
K=24 scaling (bf16 / 8-bit, qwen2-0.5b, G.143) 24/24 72/72 (bf16) / 71/72 (8-bit) 32/48 (31/48 routed) / 28/48 144/144 neighbours, 46/48 cosine route (48/48 argmax, MULTIBILLION_WSSC_SCALING.json @ 6156959)
LFM2.5-2.6B-8bit layer-13 w2 (G.143 PARTIAL) 12/12 36/36 8/24 (3/24 routed) 43/72 neighbours, 0 restore failures, kl_anchor_max=0.1185 (MULTIBILLION_WSSC_SCALING.json @ 6156959)
Prefix-marginalized W3 (bf16 / 8-bit, G.146 mixed) 11/12 30/36 11/24 (14/24 norm) / 9/24 Prefixed 9/12 -> 10/12 direct & 10 eval; 2-proto router 23/24 (PREFIX_MARGINALIZED_WSSC.json @ 6156959)

4. Two-regime hybrid agent runtime & safety (G.145, TWO_REGIME_HYBRID_RUNTIME.json @ 6156959)

TwoRegimeAgentRuntime (two_regime_runtime.py) passes 4/4 verification gates in an interleaved MLX session (26/26 CPU unit tests):

  • Regime 1 (Aligned chunked KV, CHUNK_SIZE=64, cut=64): 12/12 compliant procedural JSON, 12/12 aligned exact token match vs cold, SHA-256 tamper rejection, exact rollback trim.
  • Regime 2 (Routed 8-bit W3, tau=0.60): 12/12 direct, 23/24 dev-24, 35/36 dev-36, 15/24 held-out eval, 0 restore failures (71 apply/restore cycles).
  • Regime 3 (Deterministic external PolicyGate): 24/24 adversarial blocked, 8/8 benign + 10/10 defensive allowed (0 false positives), credential leak intercepted (interleaved_session.ok=True).

πŸ“¦ Authentic Cartridge Release Inventory (11 weight-skills/cartridge@1 Files)

All 11 published .safetensors files under cartridges/ conform to schema weight-skills/cartridge@1 (manifest.json records exact byte sizes, SHA-256 digests, ranks, and target layers):

Checkpoint Directory Cartridge File Skill / Role Rank / Layers Status Note
base_nemotron_30b/ base_nemotron_30b.aws-cso-r16.safetensors aws-cso procedural skill r=16, [24, 31, 38] Verified mount/restore
gemma4_4b/ clu-completion-r32.safetensors clu-completion persona r=32, [16, 20, 24, 28] Verified mount/restore
gemma4_4b/ gemma4_4b.aws-cso-r16.safetensors aws-cso procedural skill r=16, [19, 24, 29] Verified mount/restore
jimmy_4b/ jimmy_4b.aws-cso-r16.safetensors aws-cso procedural skill r=16, [19, 24, 29] Verified mount/restore
occamy_35b_moe/ occamy_35b_moe.aws-cso-r16-vlm.safetensors aws-cso (mlx-vlm text tower) r=16, [18, 23, 28] Verified mount/restore
occamy_35b_moe/ occamy_35b_moe.aws-cso-r16.safetensors aws-cso procedural skill r=16, [18, 23, 28] Verified mount/restore
occamy_35b_moe/ occamy_35b_moe.aws-cso-r4-vlm.safetensors aws-cso (mlx-vlm, n_probes=4) r=4, [18, 23, 28] Compact rank-4 check
proto_nemotron_30b/ proto_nemotron_30b.aws-cso-r16.safetensors aws-cso procedural skill r=16, [24, 31, 38] Verified mount/restore
thinklite_27b/ thinklite_27b.aws-cso-r16.safetensors aws-cso procedural skill r=16, [29, 37, 45] Verified mount/restore
refusal/ qwen36-35b-heretic-4bit.guardrail-r16.safetensors guardrail-strict-refusal (r=16) r=16, [18, 23, 28] Experimental negative (0/24 standalone refusal; use PolicyGate)
refusal/ qwen36-35b-heretic-4bit.guardrail-r64-L8-P8.safetensors guardrail-strict-refusal (r=64, 8L) r=64, 8 layers [18..28] Experimental negative (0/24 standalone refusal; use PolicyGate)

Note: Obsolete ungrounded files from earlier draft uploads were purged in favour of this byte-verified 11-cartridge weight-skills/cartridge@1 inventory and two_regime_runtime.py.


πŸš€ Quickstart

Hybrid two-regime runtime (aligned KV prefix + cosine-routed 8-bit W3 + external PolicyGate):

# See two_regime_runtime.py (G.145, 26/26 unit tests + 4/4 live MLX gates PASS)
from bender_bench.cartridges.two_regime_runtime import TwoRegimeAgentRuntime, PolicyGate

gate = PolicyGate()  # blocks 24/24 adversarial requests, 0 false positives
# Align prefix at CHUNK_SIZE=64 with SHA-256 verification & cold chunked tail parity,
# and dispatch routed 8-bit W3 adapters at cosine threshold tau=0.60 with exact restore.

Mounting a Dynamic Cartridge (zero-prompt approximate injection, exact unmount):

import mlx_lm
from safetensors import safe_open

# Inspect metadata & load cartridge tensors from cartridges/<model>/<file>.safetensors
with safe_open("cartridges/proto_nemotron_30b/proto_nemotron_30b.aws-cso-r16.safetensors", framework="numpy") as f:
    print(f.metadata()["weight_skills"])  # schema: weight-skills/cartridge@1

The research runtime & tests (two_regime_runtime.py, ParetoAnchorGate.strict_rsi_gate(min_samples_per_suite=50)) ship alongside weight_space_skill_compilation_paper.pdf and MEASUREMENT_ADDENDUM.md.


πŸ§ͺ Tests and scope

  • 26/26 two-regime runtime unit tests (tests/test_two_regime_runtime.py), 25/25 powered Pareto gate unit tests (tests/test_pareto_gate.py), and 250 library tests (weight-skills package); cross-model Batch 11 audit passed 99/99 checks (reports/LOOP_BATCH11_AUDIT.json @ 05d4530).
  • Benchmarks measured on Apple M4 Max (128 GB) with MLX, greedy decoding, on committed artefacts; the full protocol and provenance are in weight_space_skill_compilation_paper.pdf and MEASUREMENT_ADDENDUM.md.
  • Weight deltas change a function; they are not a standalone security boundary (0/24 refusal installed when mounted in GUARDRAIL_MECHANISM_AUDIT.json). For exact behaviour use an aligned compiled prompt cache; for safety enforcement use the external deterministic PolicyGate (24/24 blocked, 0 false positives in G.145). Benchmark promotion gates enforce n >= 50 samples per suite (G.142, ParetoAnchorGate.strict_rsi_gate(min_samples_per_suite=50) in reports/PARETO_POWERED_GATE_AUDIT.json @ 353dfb8; empirically rejecting the underpowered n=56 Gemma-4-4B pilot in G.144 -4.0% MATH-500 -> REJECTED_REGRESSION).

Citation

@article{machado2026zeroskills,
  title   = {Zero-Skills in Context: Sub-Second Algebraic Compilation of Agent Procedures into Feed-Forward Weights},
  author  = {Machado, Luis},
  year    = {2026},
  doi     = {10.5281/zenodo.24122803},
  url     = {https://doi.org/10.5281/zenodo.24122803},
  note    = {Concept DOI β€” resolves to the latest version}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support