cl33-opLM — selective reproducibility release (built for preprint v1.1; coverage against v1.3 stated below)

Paper: One Object: Memory, Navigation, and Reportability in an Operator-Only Language Model — DOI 10.5281/zenodo.22684392 · https://t3atlas.dev/cl33/paper/ · Live demo: https://cl33.t3atlas.dev Author: Garret Sutherland, MirrorEthic LLC.

This bundle contains what is necessary to independently test two of the paper's claims on frozen artifacts: the operator bottleneck (Claim 1 below) and the operator-stream transcript on the released chat checkpoint (Claim 2). It is deliberately not the training stack and it is not a reproducibility package for the whole paper: the §12 memory-organ program and every other result are documented elsewhere (see Claim coverage). Reproducibility surface ≠ complete source disclosure.

Bundle revisions

• 2026-09-09 — initial release (two checkpoints, probe, scripts, frozen eval slice), built against preprint v1.1. 09-10: model_v2.py docstring synced to the paper's §3.2 (v1.3), model card and licence added.
• 2026-10-25 — maintenance pass after an external first-pass review (M. Wright): (1) SHA256SUMS regenerated — the 09-09 manifest predated the 09-10 edits to model_v2.py and REPRODUCE.md, carried a release/ path prefix on the three weight files, and omitted README.md / LICENSE; every file in the tree is now listed by its published path. (2) Both scripts load checkpoints with torch.load(..., weights_only=True); the three .pt files contain tensors and primitive metadata only. (3) The reverse-readout probe's held-out evaluation is now reproducible on its public-corpus part: eval_slice_invert_heldout.npz ships the six public-corpus sources of the probe's own document-disjoint evaluation split and repro_reverse_readout.py --heldout recomputes the per-source rates stored in the card (the card's 45 chat rows are not distributed; see Claim 2). (4) This claim-coverage statement. No checkpoint, probe, model code or measured number changed.

Contents

file	what	sha256 (full hashes in SHA256SUMS)
cl33_oplm_prose_236m.pt	prose base, 236.5M, step 189307 (Table 1b checkpoint)	fe407328…6a1fbf
cl33_oplm_chat_236m.pt	chat/serving model, step 13996 (the cl33.t3atlas.dev model)	ddd042a6…559535
invert_probe_chat.pt	reverse-readout probe (held-out top-1 0.860, card inside)	91201342…ca79ef
eval_slice_prose_val.npy	frozen 24 × 1025-token slice of the prose-mix validation split (Claim 1 fixture)	a967e8ab…8b17ea
eval_slice_invert_heldout.npz	the probe's held-out split, public-corpus part: 108 × 1024 token ids + source ids (Claim 2 fixture)	see SHA256SUMS
model_v2.py + model.py + so33.py + wedge.py + t3v3_wedge_memory.py + tape_memory.py	model definition (load-only)	see SHA256SUMS
repro_bottleneck.py	Claim 1: the mandatory operator bottleneck	see SHA256SUMS
repro_reverse_readout.py	Claim 2: the operator stream is a transcript (demo + --heldout)	see SHA256SUMS
SHA256SUMS, LICENSE (Apache-2.0), this README.md, REPRODUCE.md

Verify the tree first: sha256sum -c SHA256SUMS (run inside the bundle directory).

Environment. Deps: torch, transformers, datasets, numpy (Python ≥ 3.10). The 2026-10-25 numbers below were re-verified on CPU with Python 3.12.3, torch 2.10.0, transformers 5.5.0, datasets 4.4.1, numpy as installed by torch; the scripts run on CPU or CUDA. Checkpoints load with weights_only=True on torch ≥ 2.4.

Claim 1 — the operator bottleneck is mandatory

Zero the emitted operators; the model loses its only path to output.

# EXACT in-domain reproduction (frozen 24x1025-token slice of the prose-mix val, shipped):
python repro_bottleneck.py --ckpt cl33_oplm_chat_236m.pt --slice eval_slice_prose_val.npy
# off-domain, public data:
python repro_bottleneck.py --ckpt cl33_oplm_prose_236m.pt
python repro_bottleneck.py --ckpt cl33_oplm_chat_236m.pt

Expected (WikiText-103 test, public data, seq 1024 — measured on this exact bundle):

ckpt	native PPL	ops-off PPL	ratio
chat + frozen slice (exact)	68.6	18,552.6	270×
prose + wikitext	≈61	≈6,500	≈106×
chat + wikitext	≈186	≈21,000	≈112×

(The paper's original 314× was a different random draw of the same validation mix; the shipped frozen slice reproduces exactly at 270×, and the claim — orders of magnitude — holds on public data too. eval_slice_prose_val.npy is derived data (GPT-2 BPE token ids) drawn from public corpora: FineWeb/FineWeb-Edu (ODC-By), DCLM, Project Gutenberg (public domain), Wikipedia (CC BY-SA), Cosmopedia (Apache-2.0), FineMath, Stack-Edu; shipped solely as an evaluation fixture with attribution.)

What this ablation does and does not isolate. Bs = 0 zeroes the emitted operators at every entry to the readout: the rotor scan becomes identity and the operator features that feed the grade tower directly are zeroed. The ratio therefore measures the completeness of the operator record as the carrier — nothing behaviourally relevant bypasses it — and not the contribution of the recurrent (scan) state. The paper's §3.2 (v1.3, Watson's independent eval-only audit of these same checkpoints) separates the two: with the operators intact, pinning the scan to identity costs 1.19–1.22×, time-shuffling the scan 1.00×, zeroing η-attention 1.00×; at LM inference the transported state is causally decorative and context reaches the output through the emitted operators. The model_v2.py docstring states the same (synced 09-10). This bundle reproduces the bottleneck row of that table; the other four cuts are reproducible on these checkpoints but their evaluator is not included here.

Claim 2 — the operator stream is a readable transcript

A probe that sees ONLY the emitted operators (no token input) decodes the text.

# demo on any text:
python repro_reverse_readout.py --text "any sentence you like"
# the held-out evaluation behind the 0.860 (recomputes the probe's stored card):
python repro_reverse_readout.py --heldout eval_slice_invert_heldout.npz

Expected: ~0.86 top-1 on typical English in demo mode (rare words fail toward semantic neighbours — that is the paper's §7 claim, not a bug).

The held-out evaluation behind the card. The probe's evaluation split is the document-disjoint tail 10 % of each of seven sources: six public corpora (18 rows × 1024 tokens each) and 45 rows of chat-template-rendered SFT conversations. The card stored in invert_probe_chat.pt is over all seven (156,672 positions): top-1 0.860, top-5 0.920, CE 1.46 bits. The 45 chat rows are not distributed (SmolTalk-derived material and in-house SFT data from a proprietary harness), so the shipped fixture is the six public-corpus sources (108 rows, 110,592 positions). --heldout recomputes those six per-source rates and the six-source aggregate:

top-1 per source	fineweb_edu	wikipedia	stack_edu	finemath	dclm	cosmopedia
stored card (2026-07-28, V100)	0.829	0.820	0.916	0.778	0.842	0.877
recomputed from this bundle (2026-10-25, CPU fp32)	0.829	0.820	0.916	0.778	0.842	0.877

Six-source aggregate top-1: recomputed 0.8436 vs card-implied 0.8437 (the card's rates are rounded to three decimals). Recomputed top-5 0.909 and CE 1.661 bits on the six sources (the card has no per-source values for these; its 0.920 / 1.46 include the chat rows, which decode at 0.899 top-1 and carry the vocabulary-extension specials). Receipt for the whole split: on 2026-10-25 the full seven-source split was recomputed internally with this same script and matched the card to four decimals (0.8598 / 0.9203 / 1.4617); that run's log is in the project tree, not in this bundle.

eval_slice_invert_heldout.npz is derived data (GPT-2 BPE token ids) from the held-out tail region of the same public shards as the training mix — FineWeb-Edu (ODC-By), Wikipedia (CC BY-SA), Stack-Edu (per-repository licences), FineMath (ODC-By), DCLM (CC-BY-4.0), Cosmopedia (Apache-2.0) — shipped solely as an evaluation fixture with attribution. The probe decodes [Bs|Bq|Bk] (1,440 floats per position) with a 1,440→2,048→50,262 MLP; features are captured in float16 exactly as at training.

Claim coverage — what this bundle reproduces, and what it does not

Numbering follows preprint v1.3 (2026-09-10, the live version at the DOI). The bundle was built on 2026-09-09 against v1.1; none of the numbers it reproduces changed between v1.1 and v1.3. What changed in v1.2/v1.3 was the paper's statement of the scan's causal share (new §3.2; abstract and §13 wording), which this bundle does not test beyond the bottleneck row.

paper claim	status in this bundle
Mandatory operator carrier — ablation ratio on the serving checkpoint (§1, §2, Appendix R; abstract "mandatory carrier")	Reproduced: repro_bottleneck.py, 270× exact on the shipped slice, ≈106×/≈112× on WikiText-103. Completeness of the record; does not localize scan vs. direct operator features.
§3.2 five-condition grid (scan pinned / shuffled / η off) on these checkpoints	Reproducible on these artifacts, evaluator not shipped. Independently reproduced by N. Watson from this bundle's checkpoints (paper §3.2).
Contribution 4, §7 — inversion battery	Partially reproduced. Held-out decoding of the chat checkpoint's operators is reproduced here per source on the public part of the probe's split (six sources, aggregate 0.844; the card's 0.860 includes 45 undistributed chat rows). The §7 numbers as printed — 0.86 top-1 on scale-C, +4.7 bits beyond prefix predictability, 80.7 % verbatim document reconstruction, substitution identity — were measured on the scale-C 163M model with the inversion-battery scripts; scale-C is not released.
Table 1b — bits-per-byte parity of the prose base (§10)	Artifact shipped, harness not. cl33_oplm_prose_236m.pt is the Table 1b checkpoint; the lm-eval adapter and right-padding rule are in the project tree (Appendix R), not here.
Contribution 1 (two memories, §3), 2 (navigation parity, §4), 3 (reversible tape, §5), 5 (claims ladder, §8), 6 (Jacobian lens, §9; §3.1 lever probe), 7 (capability battery, §6), 8 (matched tax, §10 Table 1), §9.6 regime sensor	Documented elsewhere, not reproducible from this bundle. Scripts, checkpoints and dated work-log records are listed in Appendix R.
§12 memory-organ program	Not included; labelled ongoing in the paper.
Scan/tower training controls (R0 / R1 no-tower / R2 identity-scan-from-init, probe scale)	Not in the paper and not here; reported in the 2026-09-13 results packet to N. Watson.

If you are checking the paper against this bundle, the two rows marked Reproduced and Partially reproduced are the claims it can settle. Everything else is a pointer.

Training data statement

Four stages, ~11.8B tokens cumulative (GPT-2 BPE; chat stages use a 5-token spliced extension → vocab 50262). cl33_oplm_prose_236m.pt is the Stage-2 endpoint; cl33_oplm_chat_236m.pt (the cl33.t3atlas.dev demo model) is the Stage-4 endpoint.

Stage 1 — from-scratch pretrain, 5.000B tokens ("ultimate_mix"): FineWeb-Edu 2.00B (40%) · DCLM 1.00B (20%) · Cosmopedia 0.50B (10%) · FineMath 0.50B (10%) · Stack-Edu 0.50B code (10%) · Wikipedia 0.50B (10%). All 56 shards sha1-fingerprinted in the project's run manifest.

Stage 2 — context splice 512→1024 + prose continuation, 6.20B tokens ("prose_mix"): FineWeb 35% · Gutenberg 15% · Wikipedia 10% · DCLM 10% · FineWeb-Edu 10% · Cosmopedia 8% · FineMath 6% · Stack-Edu 6%. This is the checkpoint whose bits-per-byte matches token-matched Pythia-160m (paper, Table 1b).

Stage 3 — chat SFT, ~0.43B tokens, 2 epochs: 327k chat conversations (SmolTalk-derived + a deduplicated diverse set + 5,472 in-house self-Q&A pairs + 120 persona seeds), 61k self-corpus document chunks, 10k pretrain-replay documents (forgetting guard), 427 reasoning traces (287 R1-derived CoT + 140 general; system-prompt-gated).

Stage 4 — pinned-persona SFT, ~0.19B tokens: identity trained as a pinned tape record (authority channel) rather than system-prompt tokens; in-house synthetic persona corpus.

Sources are public corpora (FineWeb/FineWeb-Edu ODC-By; DCLM CC-BY-4.0; Cosmopedia Apache-2.0; Wikipedia CC-BY-SA; Gutenberg public domain; SmolTalk Apache-2.0; Stack-Edu per-repository licenses; FineMath ODC-By) plus in-house synthetic material (self-Q&A, persona, general-think; the 287 CoT traces are R1-distilled). No private or user data in any stage. The shipped eval_slice_prose_val.npy is a 24×1025-token fixture drawn from the Stage-2 validation split; eval_slice_invert_heldout.npz (public-corpus rows only) is described under Claim 2.

What is NOT here, and why

Training orchestration, data pipelines, the §12 memory-organ program (labeled ongoing in the paper), and downstream control/steering machinery. The claims those support are either reported with their own dated work-log provenance (paper, Appendix R) or not yet published. This bundle is scoped to verify what the preprint asserts about these frozen artifacts — the two claims above — and the coverage table says which.

Checksums / provenance

Both checkpoints are weights-only exports (optimizer state stripped) of the exact training checkpoints named in the paper's Appendix R; the released chat checkpoint is tensor-for-tensor identical to the checkpoint the probe was trained on. Verify with sha256sum -c SHA256SUMS inside the bundle directory. Loading: torch.load(..., weights_only=True) in both scripts; the .pt files hold state dicts plus config (dict of primitives), step (int) and, for the probe, its evaluation card.