GitHub · Documentation · Working paper · NCM paper · Replay demo

NCM-TinyWorld-5M-MLX

The quickstart model of ncm-mlx: a 5.0M-parameter Next-Concept Modeling generator, a 1.3M-value multiscale residual-VQ codec, and a same-size autoregressive baseline (4.8M), all trained on TinyWorld, a synthetic language of 15,360 sentences like ann sees two red cats. (32 characters). The generator writes a sentence in 6 coarse-to-fine levels (1, 2, 4, 8, 16, 32 code positions, one generator call each) plus one codec decode, instead of 32 sequential character steps. The whole pipeline trains in about 5.5 minutes on an Apple Silicon laptop.

A teaching-size model on a toy language. At this size the NCM is less accurate than the equal-size AR baseline (89.8% against 98.6% valid sentences at T = 1.0). It shows how the method works; it says nothing about real text.

Previously published as janmusic/ncm-mlx-tinyworld; the old name redirects here. Method by Kumbong et al. (arXiv:2610.14162); this is an independent implementation, not affiliated with HazyResearch or Stanford, and not compatible with the released NCM checkpoints.

At a glance

Generator	4,972,160 parameters, block-diagonal attention, no prompt conditioning (best validation step 1,500 of 3,000)
Codec	1,301,248 saved values: contextual transformer encoder, LayerNorm, six residual VQ levels, transformer decoder
AR baseline	4,752,921 parameters, character level, same width and depth
Tokenizer	characters, padded to 32 positions
Hierarchy	6 levels: 1, 2, 4, 8, 16, 32
Format	float32 safetensors, loaded by ncm_mlx

Quick start

The CLI reads weights from artifacts/toy-grammar-12k/ (the out_dir of configs/toy-grammar-12k.json, the CLI's default config). Needs an Apple Silicon Mac (or Linux with MLX CUDA) and uv.

git clone https://github.com/janmus1c/ncm-mlx.git && cd ncm-mlx
uv sync --locked --python 3.12  # Apple Silicon; Linux NVIDIA: add --group cuda
uvx --from huggingface-hub hf download janmusic/NCM-TinyWorld-5M-MLX --local-dir artifacts/toy-grammar-12k
uv run python -m ncm_mlx demo --config configs/toy-grammar-12k.json --n 3   # level-by-level samples
uv run python -m ncm_mlx eval --config configs/toy-grammar-12k.json        # quality metrics (JSON)
uv run python -m ncm_mlx baseline --config configs/toy-grammar-12k.json --bench   # uses ar.safetensors

These commands need a GPU (Metal or CUDA); the CPU is only used for the repository's exactness tests. Sampling temperature, seed and top-k are CLI options. The banner shows the output of demo --seed 0 with --n 4, all four samples.

Results

Experiment E1 on an Apple M5 Pro (64 GB, MLX 0.32.3) with other GPU services running. Validity is graded by the TinyWorld parser; 2 × 512 samples per temperature.

	NCM generator	AR baseline + KV cache
Valid sentences, T = 1.0	89.8%	98.6%
Valid sentences, T = 0.7	95.8%	99.8%
Number agreement when well formed	99.8%	100%
Valid and not in training set, T = 1.0	16.3%	23.5%
Model calls per sequence	6 + 1 decode	32
Latency p50, batch 1 / 16 / 256	4.5 / 5.8 / 37.8 ms	30.5* / 22.9 / 49.8 ms

Codec: 100% of held-out sentences rebuilt exactly, 100% codebook use at every level (codec and generator quality are separate numbers). *The AR batch-1 p50 was 19.8 ms in an otherwise identical earlier run; small-batch timings depend on GPU load. Unseen sentences are 16.4% of the language, so both models are near the rate expected from sampling the whole language.

How it works

Honest limits

• A synthetic, tiny language. No open-domain text, no prompts, no instruction following. Quality on real text is unknown.
• Worse than AR at equal size. About 9 points lower validity at T = 1.0 (about 4 at T = 0.7). Errors are mostly local spelling slips (ret fox); the sentence-wide agreement rule is almost always kept.
• Latency depends on load. The latency advantage varies with GPU contention, and neither sampler is compiled. The upstream paper's 10–31x speedups compare NCM with diffusion language models on GPUs; nothing here claims a speedup over large LLMs.
• Numerics. fp32 matmuls on Metal are reduced precision (about 1e-3 relative error).

Files

File	Contents	Size
codec.safetensors, codec.json	Codec (6 levels) and tokenizer	5.2 MB
generator.safetensors, generator.json	Next-concept generator (best validation step 1,500 of 3,000)	19.9 MB
ar.safetensors, ar.json	Character-level autoregressive baseline, same width and depth	19.0 MB
config.json	Run config (configs/toy-grammar-12k.json in the repository)
assets/	Banner and figures for this card

Training

Codec 52 s, generator 266 s (3,000 steps, best-validation checkpoint), AR baseline 134 s, on the hardware above. Training data: 12,000 generated sentences; 1,000 validation; 2,360 never seen. Total about 5.5 minutes with mise run train. These are the toy-grammar-12k artifacts of experiment E1.

Related models

Model	What it is	Provenance
NCM-TinyStories-87M-MLX	Best NCM text model trained in this project	Trained here
AR-TinyStories-91M-MLX	Autoregressive baseline for the 87M NCM	Trained here
NCM-TinyStories-89M-Upstream-MLX	Released TinyStories checkpoint, converted to MLX	Port, weights by HazyResearch
NCM-Sudoku-25M-Upstream-MLX	Released Sudoku checkpoints, converted to MLX	Port, weights by HazyResearch
NCM-Sudoku-6.4M-MLX	Small Sudoku model	Trained here
NCM-TinyWorld-5M-MLX	Quickstart model on a synthetic grammar (this model)	Trained here

All of them are in the Next-Concept Modeling in MLX collection.

Citation

@misc{kumbong2026ncm,
  title         = {Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation},
  author        = {Hermann Kumbong and Ishan S. Khare and Iddah Mlauzi and Andrew Woen and Charles Ding and E. Kelly Buchanan and Simran Arora and Azalia Mirhoseini and Christopher R\'e},
  year          = {2026},
  eprint        = {2610.14162},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.14162}
}

@software{music_ncm_mlx,
  title   = {{NCM-MLX}: Next-Concept Modeling in {MLX}},
  author  = {Music, Jan},
  year    = {2026},
  version = {0.4.0},
  doi     = {10.5281/zenodo.23781233},
  note    = {Version 0.4.0: doi:10.5281/zenodo.23953482},
  url     = {https://github.com/janmus1c/ncm-mlx}
}

@misc{music_ncm_mlx_paper,
  title     = {Coarse-to-Fine Text Generation on a Laptop: A Native {MLX} Study of Next-Concept Modeling},
  author    = {Music, Jan},
  year      = {2026},
  version   = {0.13},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23953213},
  note      = {Working paper, not peer reviewed}
}

The software DOI 10.5281/zenodo.23781233 resolves to the latest version; 10.5281/zenodo.23953482 is version 0.4.0. These weights were first published with an earlier version of the software.

License

Code and weights: MIT. TinyWorld is generated by the repository; no external data is used.