Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
danielhanchen 
posted an update 4 days ago

The 94.64% is the IQ1_M row on its own.

91.88 / 97.08 = 94.64. The other two GSQ-RCO quants keep 97.86% (IQ2_XS) and 98.07% (IQ2_XXS) of your Q8 reference on AIME25. All three averaged: 96.86%.
Still short of 100%, so the non-reproduction holds. But the headline reads like the average, and it is the worst of three.

The row I would build the post around is UD-Q2_K_XL beating its own Q8 reference.
AIME25: 468 vs 466 of 480 answers. HMMT Feb26: 369 vs 364 of 528.
16 samples on 30 problems is still 30 problems, so a quant beating its reference is a direct read on the noise floor.
The size-matched gaps sit above it: UD-IQ1_M over GSQ IQ2_XXS by 1.87 on the avg of 6 (250.4 vs 248.8 GB), UD-IQ1_S over GSQ IQ1_M by 1.75 (233.0 vs 231.3 GB).

Did you run the Q8 reference twice on different seed bases? That replicate gap is the number that separates a quant effect from a seed effect here.