divagr1925/Qwen3.5-4B-Qwenputer-CHIP8

A computer, compiled into the weights of a language model.

This is the CHIP-8 Qwenputer. The main model, with a RISC-V CPU in its weights, is divagr1925/Qwen3.5-4B-Qwenputer.

Code on GitHub · Base model: Qwen/Qwen3.5-4B · Apache-2.0

Qwenputer is Qwen3.5-4B with a working computer compiled directly into its weights. It isn't a plugin, a tool or a sandbox. It's a CPU, a big-number maths library and its memory, written as transformer layers and appended to the stock model. None of its weights are learned; every one is computed analytically by a compiler.

We use the model's own Gated DeltaNet state as the computer's RAM. Qwen3.5's Gated DeltaNet layers carry a running state that the delta rule can overwrite one row at a time, exactly, so each row works as a memory cell. The computer's registers, memory and program counter all live there, inside the model's own attention state.

So when Qwenputer writes 48213 × 9917 =, the answer isn't a guess. The computer in its weights runs the multiplication, and the model writes 478128321 itself, digit by digit, exactly. It has never printed a wrong answer, from 3-digit sums to 20-digit products, and it loads like any other Qwen checkpoint, with no custom code and no trust_remote_code.

Benchmarks

These are greedy decoding scores. Each row compares base Qwen3.5-4B and Qwenputer on the same problems. The Python-tool column is base Qwen calling a real Python interpreter on the host, which is the ceiling for this kind of help.

Benchmark	Base Qwen3.5-4B	Qwenputer	Python tool (host)
NUPA integer add / sub / mul, 3-20 digits (576)	48.6	100.0	100.0
DeepMind arithmetic, 6 sets pooled (600)	80.8	92.3	94.2
DeepMind div_big (100)	77	97	85
DeepMind mul_big (100)	66	93	91
DeepMind clean big add / mul (200)	69.5	99.0	98.5

The computer has never printed a wrong result. We replayed about 3.9 million positions of model output through a reference simulator of the circuit and found zero mismatches. MMLU: 71.9 vs 71.7 for base (99% identical answers, KL 7e-4).

GSM8K and GSM-Hard are ties, because word problems are limited by reasoning rather than arithmetic. On long DeepMind expression chains, base Qwen is still ahead: Qwen tends to re-verify exact results until it runs out of its token budget.

The rows are the best mode per benchmark across our development checkpoints. This checkpoint contains all the features behind them.

Quickstart

git clone https://github.com/divagr18/qwenputer && cd qwenputer && uv sync --extra hf
hf download divagr1925/Qwen3.5-4B-Qwenputer-CHIP8 --local-dir qwenputer-4b
uv run python -m chipformer.q35.demo qwenputer-4b

Every prompt starts with the computer's BIOS. That's a boot record of chip tokens, which loads the arithmetic library into the computer's memory, followed by Qwen's normal chat template. After that it's a plain generate() call:

import json, torch
from transformers import AutoModelForImageTextToText, AutoTokenizer
from chipformer.q35.demo import TOOLS
from chipformer.q35.lib import bcd
from chipformer.ref.emu import Chip8
from chipformer.trace import boot_tokens

torch.backends.cuda.matmul.allow_bf16_reduced_precision_reduction = False  # keep matmuls exact

path = "qwenputer-4b"
tok = AutoTokenizer.from_pretrained(path)
model = AutoModelForImageTextToText.from_pretrained(path, dtype=torch.bfloat16, device_map="cuda")
off = json.load(open(f"{path}/chipformer.json"))["chip_token_offset"]
bios = [t + off for t in boot_tokens(Chip8(bcd.ROM, bcd.QUIRKS).cells)]

msgs = [{"role": "user", "content": "What is 48213 * 9917?"}]
ids = bios + tok.apply_chat_template(msgs, tools=TOOLS, add_generation_prompt=True,
                                     enable_thinking=False)["input_ids"]
out = model.generate(torch.tensor([ids], device="cuda"), max_new_tokens=4096, do_sample=False,
                     cache_implementation="static")
print(tok.decode(out[0, len(ids):], skip_special_tokens=True))

The tools, the system prompts that work best and the optional decoder settings are documented in docs/running.md.

How it works

Qwen writes        "48213 × 9917 ="
computer layers    read the digits, run the BIOS, emit trace tokens
the model prints   " 478128321"
Qwen continues     as if it had written the number itself

Running. Each generated trace token is one write to the computer's memory. A few tokens make one instruction, so the trace is the machine's execution log. The MLPs do the arithmetic as exact lookup tables, and every value stays exact in bf16.

The bridge. The computer layers sit before Qwen's and read Qwen's tokens: digits, operators, tool-call tags. When a calculation is complete they run it and print the answer as normal text. Qwen's state is frozen while the computer runs.

What it can do. The computer is about 0.7M nonzero weights covering add, subtract, multiply, exact division (reduced fractions when it doesn't terminate), signed numbers, mod, gcd, and whole expressions with brackets, powers and decimals. Division that doesn't terminate prints as a reduced fraction.

Gated DeltaNet as RAM

The computer's RAM is the state of Qwen3.5's own Gated DeltaNet layers. Qwen3.5 mixes full attention with these layers, and each DeltaNet head keeps a state matrix that it updates at every token with the gated delta rule (panel a). In the computer's heads the gate is held open (α = 1, so nothing fades) and the write strength is β = 1. Each key is a one-hot vector picking a single row, so every update replaces exactly that row with the value and leaves the rest untouched. That's a RAM write. A RAM read is the same head's query: a one-hot query returns exactly that row (panel b).

The rest is layout (panel c). The low bits of an address choose the head and the remaining bits choose the row, and each row stores one byte as bits. The machine's memory is spread over a few dozen heads, and its registers, program counter and call stack each get heads of their own. Each key head drives two value heads, which doubles the capacity. The gate is useful too: α = 0 for one token wipes a whole head (that's how the screen is cleared), and α = ½ makes a head halve every step, which the circuit uses as a counter.

Writes arrive through the token stream, so the trace is the computer's execution log. An ADDR token names the row, the VAL token after it carries the value, and STEP ends the instruction. Nothing here needed a new layer type: the computer's memory is the same Gated DeltaNet recurrence Qwen3.5 already uses for language.

Is there really no tool call?

There is no tool. Qwen can write a calculation inline (48213 × 9917 =), or use the tool-call syntax it was trained on (<tool_call><function=mul>…). In both cases nothing outside the model reads it. The <tool_response> that follows is predicted by the model's own layers, token by token, exactly like every other token. The computer's work shows up in the output as a burst of special trace tokens, which skip_special_tokens=True hides.

Model details

Base	Qwen/Qwen3.5-4B, Qwen's own weights unchanged
Architecture	stock Qwen3_5ForConditionalGeneration; config-only changes
Computer	12 compiled layers before Qwen's 32; CHIP-8 machine plus bignum BIOS
RISC-V version	divagr1925/Qwen3.5-4B-Qwenputer
Hidden size	3200 (Qwen's 2560 plus the computer's dims)
Size	14.65 GB in bf16 (7.3B stored parameters: Qwen3.5-4B's 4.7B, plus zero padding and the computer's layers, of which only 0.7M weights are nonzero)
Precision	exact in bf16

Citation

@software{agrawal2026qwenputer,
  author  = {Agrawal, Divyansh},
  title   = {{Qwenputer}: a computer compiled into the weights of a language model},
  year    = {2026},
  version = {0.1.0},
  url     = {https://github.com/divagr18/qwenputer},
  license = {MIT}
}

License

Apache-2.0. This is a modified version of Qwen/Qwen3.5-4B: the config is widened and computer layers are added, but Qwen's own weights are unchanged.