prism-ml/Ternary-Bonsai-2-27B-gguf

Independent test: 98% retention on easier tasks, 87% on harder reasoning/coding (vs 4-bit Qwen3.8 27B)

🔥❤️ 7

#102 opened 9 days ago by

heldoutlab

Ternary-Bonsai-2-27B-PQ2_0: Comprehensive Model Evaluation and User Review

#101 opened 11 days ago by

lscdev2

Speculative Drafters for Ternary-Bonsai-2-27B (11~13 tok/s on Apple Silicon M3)

🔥 1

#100 opened 13 days ago by

NOhhh

Has anyone compared Ternary Bonsai 2 model quality retention versus ISTA-DASLab's GSQ-RCO?

#99 opened 14 days ago by

ssvb

RTX 3060 12GB measured: 37.6 tok/s, 626 tok/s prefill, 9.1 GB at 32K

❤️ 1

#98 opened 15 days ago by

nazeerAilabs

Please someone fork this with Kimi Delta Attention as well!

#97 opened 17 days ago by

N7Shepard

Crazy fast, already optimized to over 70 tokens/s, can be pushed up to 300 tokens/s, on an 8GB 4060.狂飙，已经被优化到70token/s以上,修改最高300token/s，在8G的4060

👍🔥 1

#96 opened 17 days ago by

Request to bonsai

👀👍 7

#95 opened 17 days ago by

Banaxi-Tech

Bonsai 2 in Strata: 64.4 tok/s on one RX 7900 XT with no MTP.

#94 opened 19 days ago by

thank you very much

🔥 2

#92 opened 21 days ago by

zeroyet

Llama.cpp fork update regression?

👀 2

#91 opened 22 days ago by

ysf78

bonsai version of qwen 3.8 flash next and qwen3.8-27b but swift version

#90 opened 22 days ago by

trombonator

RTX 2060 SUPER 8GB: Bonsai 2 27B PTQ1_0 jumps from 16.76 to 39.31 tok/s after a llama.cpp update (no MTP)

#89 opened 22 days ago by

mktnhr

prism-llama-cli update above b10743 is bad for SVG!

#88 opened 23 days ago by

Independent results for Bonsai-2 PQ2_0 (GSM8K / MATH / HumanEval) with caveats

#87 opened 24 days ago by

It did not met my expectation (No it actually great)

#86 opened 28 days ago by

emircanerkul

Bonsai 3

👍 6

#84 opened 28 days ago by

jezzza1401

−2 logit bias on PTQ1_0, RTX 5060 Laptop: MATH-500 44/50 → 43/50

#83 opened about 1 month ago by

7dollarbooks

Add RTX 3080 (10 GB) to the cross-platform throughput table

#82 opened about 1 month ago by

Siphion

Works on Tesla V100-16GB (SM 7.0): 52 tok/s @ 64K context

#81 opened about 1 month ago by

Vyach59

nobody told us, but Pelikan can be ANIMATEEED!

#80 opened about 1 month ago by

could gemma4-31B-it be better?

👍 2

#79 opened about 1 month ago by

luckystar199746

Intel Arc: a faster SYCL path for PTQ1_0 (about 88 t/s with MTP on a B580)

#78 opened about 1 month ago by

torchit1

Sitting frozen like a nuclear bomb, the 8G laptop speeds up from 50 tokens straight to 70 tokens，8G再次从50Token/S大幅升到恐怖的70Token/s

#75 opened about 1 month ago by

I made a ternary 35B-A3B based on the results of my previous Bonsai-2 forensics & research work.

🔥 5

#74 opened about 1 month ago by

Think too much

#73 opened about 1 month ago by

jezzza1401

Which Jinja Template?

#70 opened about 1 month ago by

Omnomynous

image to SVG test: Ichigo :)

🔥 4

#69 opened about 1 month ago by

Does not work in LM Studio (NB - update CUDA, use NOXZURE1's answer)

#68 opened about 1 month ago by

SoulFireMage

Memory performance

👀 1

#67 opened about 1 month ago by

trombonator

One 27B model, two jobs — chat:50token/s + JEV decision engine:0.5s — on an 8 GB laptop

🧠 3

#65 opened about 1 month ago by

Can you create Ternary versions for 35b-a3b models?

#64 opened about 1 month ago by

naildirect

Pull coding capability, the Tetris.html which Bonsai-2-27b generated cannot play - console error

#63 opened about 1 month ago by

mind0n

I took Bonsai 2 27B apart, part 2: the format is public, the last 8% is ordinary ternary QAT.

👀 1

#62 opened about 1 month ago by

Set the model name and min_p in the GGUF metadata (weights unchanged)

#59 opened about 1 month ago by

bri-prism

for 16gb vram context size <=114688 is fast. Pelican is gorgeous :)

🤯👍 4

#56 opened about 1 month ago by

intel alchemist kernel

#55 opened about 1 month ago by

Demilenos

Real KLD testing against Unsloth's gguf BF16 of Qwen 3.8-27B

👀👍 4

#54 opened about 1 month ago by

mrumel

When will they release the official version with ROCM or Vulkan support for the 6900XT?

#53 opened about 1 month ago by

aidoluiz

AMD works just wonderfully, here is how:

❤️ 1

#52 opened about 1 month ago by

RegisteredWednesday

issue : 6800 (rdna 2) gpu card on PrismML-Eng llama.cpp . cant run model by unsloth Studio and ..

#51 opened about 1 month ago by

myhugginfacegacc

SYCL backend: any speculative type collapses performance (even target prefill drops ~200x) - draft model itself is healthy

#50 opened about 1 month ago by

Yoo00ooOO

How does the 2080 Ti 22G perform with this model?

#49 opened about 1 month ago by

coresen

PLEASE stop lying about the "intelligence" of the model.

🔥👍 28

#47 opened about 1 month ago by

Splarkszter

在4060笔记本，8G下，优化到45Token/S之后，再提10%到50Token/S，但这还不是上限...理论上限可能高达65Token/S

👍 2

#46 opened about 1 month ago by