rina-llama.cpp
A llama.cpp build for ternary GGUF models on x86 CPUs, maintained by Rinstitute. It is the
PrismML llama.cpp fork (branch prism, which tracks upstream
llama.cpp) plus tq16, a set of CPU kernels for the three ternary formats used by
Rina Juvenile Exp: PQ2_0, PTQ1_0 and Q2_0.
- 中文:面向 x86 CPU 的 llama.cpp 构建,为三值 GGUF(PQ2_0 / PTQ1_0 / Q2_0)加了 AVX2 / AVX-VNNI / AVX-512 内核。
- 日本語:三値 GGUF(PQ2_0 / PTQ1_0 / Q2_0)向けに AVX2 / AVX-VNNI / AVX-512 カーネルを追加した、x86 CPU 用の llama.cpp ビルドです。
Release 2026.11.14
| File | Contents |
|---|---|
rina-llama.cpp-2026.11.14-linux-x64.tar.gz |
Linux x86-64, statically linked (any distribution): llama-cli, llama-completion, llama-server, llama-bench |
rina-llama.cpp-2026.11.14-src.tar.gz |
Full source of this release |
source/0001-*.patch, source/0002-*.patch |
The tq16 changes as patches on top of PrismML prism 6447906 |
SHA256SUMS |
Checksums |
Base: PrismML prism at 6447906bf96c1a023980d3453fb59daa6af3bb23; tq16 at de0d48e42aaaaae3bad1cc2d44e1bcf8a80a3867.
Windows builds are not included yet.
Quick start (Linux)
tar xzf rina-llama.cpp-2026.11.14-linux-x64.tar.gz
cd rina-llama.cpp-2026.11.14-linux-x64
./llama-cli -m Rina-Juvenile-Exp-2B-PQ2_0.gguf
./llama-server -m Rina-Juvenile-Exp-2B-PQ2_0.gguf --host 127.0.0.1 --port 8080
The top-level llama-* files are small launchers. They read /proc/cpuinfo and run one of three builds in bin/:
| Level | Requires | Typical CPUs |
|---|---|---|
avx512 |
AVX-512 F/CD/BW/DQ/VL + VNNI | Intel Cascade Lake, Ice Lake, Sapphire Rapids and later Xeons; AMD Zen 4, Zen 5 |
avxvnni |
AVX2, FMA, F16C, BMI2, AVX-VNNI | Intel Alder Lake and later Core |
avx2 |
AVX2, FMA, F16C, BMI2 | Intel Haswell and later; AMD Zen 1–3 |
RINA_CPU_LEVEL=avx2 (or avxvnni, avx512) forces a level. CPUs without AVX2 are not supported by this
package; use an upstream llama.cpp build there.
What tq16 changes
- At load time the weights of all three formats are rearranged into one layout of 16-column tiles. Each 64-byte
line holds 2-bit codes such that
(line >> 2s) & 3is a 16-column × 4-input operand forvpdpbusd(AVX-512 VNNI, AVX-VNNI) orpmaddubsw(plain AVX2). Every output column accumulates in its own SIMD lane. - PTQ1_0 trits are expanded to 2-bit codes when the model loads.
- Activations are quantized to int8 with one scale per 32 values.
- The output of a token does not depend on how many tokens are in the batch, so speculative decoding reproduces plain decoding bit for bit.
Environment variables:
| Variable | Effect |
|---|---|
GGML_TQ16=0 |
Use the original PrismML kernels |
GGML_TQ16_ACT=128 |
One activation scale per weight group instead of per 32 values: about 10% faster prompt processing on VNNI CPUs, slightly lower precision (KLD 1.46e-4 instead of 1.02e-4) |
Speed
Rina Juvenile Exp 2B, llama-bench -p 512 -n 128, tokens per second. “PrismML” is the same source with
GGML_TQ16=0. The avxvnni and avx2 rows are the same CPU running builds without AVX-512.
2-vCPU virtual machine (host Ryzen 9 9950X, about 20 GB/s memory bandwidth), 2 threads:
| Format | Level | PrismML pp512 | tq16 pp512 | PrismML tg128 | tq16 tg128 |
|---|---|---|---|---|---|
| PQ2_0 | avx512 | 195 | 377 | 49.3 | 54.0 |
| PQ2_0 | avxvnni | 136 | 258 | 35.1 | 53.2 |
| PQ2_0 | avx2 | 92 | 163 | 30.2 | 55.9 |
| PTQ1_0 | avx512 | 58 | 387 | 28.6 | 53.2 |
| PTQ1_0 | avx2 | 40 | 177 | 22.8 | 55.0 |
| Q2_0 | avx512 | 56 | 371 | 27.0 | 49.9 |
| Q2_0 | avx2 | 53 | 167 | 25.5 | 49.4 |
On this machine, generation at 2 threads is close to the memory-bandwidth limit.
Ryzen 9 9950X, all cores (16 cores / 32 threads, dual-channel DDR5-6400, Linux under WSL 2, this release
package, avx512 level):
| Format | PrismML pp512, 16 threads | tq16 pp512, 16 threads | PrismML tg128 | tq16 tg128 |
|---|---|---|---|---|
| PQ2_0 | 827 | 1575 | 94.9 | 98.5 |
| PTQ1_0 | 325 | 1659 | 102.3 | 95.5 |
| Q2_0 | 311 | 1759 | 91.0 | 95.7 |
- tg128 is the median of 10 samples at the best thread count, which was 8 to 12 threads.
- Generation is limited by memory bandwidth here (about 51 GB/s), so thread counts from 6 to 16 give similar results.
- Using all 32 threads (SMT) is slower than 16 for both prompt processing and generation.
- With a 4K context (
-d 4096, PQ2_0, 16 threads), tq16 gives pp512 1447 and tg128 82.4.
Single core of the same CPU:
| Format | Level | PrismML pp512 | tq16 pp512 | PrismML tg128 | tq16 tg128 |
|---|---|---|---|---|---|
| PQ2_0 | avx512 | 111 | 229 | 37.7 | 64.9 |
| PQ2_0 | avx2 | 59 | 99 | 24.8 | 55.4 |
| PTQ1_0 | avx512 | 33 | 232 | 22.2 | 65.7 |
| Q2_0 | avx512 | 30 | 225 | 20.5 | 66.2 |
Accuracy
KL divergence against an F16 copy of the same weights (exact for ternary weights × fp16 scales), 40 chunks of 512 tokens of Chinese / Japanese / English chat text:
| Kernel | KLD | Same top-1 token |
|---|---|---|
| PrismML, AVX-512 | 1.00e-4 | 99.45% |
| PrismML, AVX2, PQ2_0 | 1.66e-4 | 99.37% |
| tq16, all levels | 1.02e-4 – 1.03e-4 | 99.43% – 99.53% |
A unit test (tests/test-tq16.cpp in the source) checks matrix products against a double-precision reference:
300 shapes per format, each with two activation distributions.
Known limitations
- Only x86-64 Linux binaries so far. tq16 does nothing on ARM; there, PrismML's own kernels are used.
- PTQ1_0 uses as much memory as PQ2_0 after loading, because its trits are expanded to 2-bit codes. On machines
with many cores and limited memory bandwidth, PTQ1_0 generation can be a few percent slower than with
GGML_TQ16=0: about 5% on the 9950X at 8 or more threads. llama-serveris built without TLS. Put a reverse proxy in front of it if it must be reachable from other machines.
Building from source
tar xzf rina-llama.cpp-2026.11.14-src.tar.gz && cd rina-llama.cpp-2026.11.14-src
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_NATIVE=ON
cmake --build build -j --target llama-cli llama-server
To apply the patches to a newer PrismML prism checkout, run git am source/*.patch.
License
MIT, the same as llama.cpp. Copyright (c) 2023-2026 The ggml authors; the tq16 changes are by Rinstitute. The
licenses of bundled third-party code are in licenses/ inside each archive.