IsValorum PRO
IsValorum
AI & ML interests
I'm tired of generic quantizations that consume too many resources when they could be even higher quality and consume much less, but nobody was doing it then I thought Why don't I do it myself? And here I am, the first person to dedicate everything to quality over quantity.
Recent Activity
updated a model about 4 hours ago
IsValorum/Cyber-Tiel-Coder-35B-A3B-APEX-I-NanoPlus-GGUF updated a model about 5 hours ago
IsValorum/Cyber-Tiel-Coder-35B-A3B-APEX-I-MiniPlus-V2.1-GGUF published a model about 5 hours ago
IsValorum/ThinkingCap-gemma-4-26B-A4B-it-BF16-GGUFOrganizations
None yet
replied to their post 10 days ago
replied to their post 10 days ago
Keeping router weights in F32 avoids quantizing those tensors, while upstream quantization can still change the hidden states they receive. In the LFM2.5 MiniPlus card, does 'zero routing drift' mean measured expert-selection agreement with BF16, or that the router tensors are unchanged? That distinction would help interpret the precision map.
The main idea is to preserve the original expert-selection mechanism by keeping the routers and their biases in F32. Given identical hidden states, the preserved routers will select the same experts as the original model.
The quantized experts can still introduce small differences in their outputs, which can affect the final token probabilities and occasionally change the generated token. However, a different token doesn't necessarily mean an incorrect one. Even with aggressive quantization such as IQ3_XXS, the model can still produce valid and coherent outputs.
The goal is to preserve routing fidelity while minimizing the impact of expert quantization on overall generation quality.
posted an update 14 days ago
Post
3,711
Navigating the APEX-I and VAL-APEX Quantization Ecosystem on Hugging Face
As the catalog approaches 400,000 total downloads across 69 releases, all models are now structured into dedicated official collections on Hugging Face.
Our goal remains consistent: eliminate the compromise between memory overflow and cognitive collapse on consumer hardware (especially 16 GB VRAM GPUs like the RTX 4080, RTX 4070 Ti Super, and RX 7800 XT).
Instead of uniform block quantization, all APEX builds use per-tensor mixed precision guided by 4-moment Importance Matrix statistics:
- MiniPlus Tier (approx. 14 to 14.8 GiB): Strict 0% 2-bit policy. All MoE routed experts locked at minimum IQ3_XXS with Q4_K/Q6_K attention. Designed for 24 GB setups or 16 GB setups with moderate context.
- NanoPlus Tier (approx. 11.5 to 12.3 GiB): Strategic use of IQ2_S/IQ2_XXS exclusively in core feedforward blocks, keeping routers and attention high-precision. Leaves ample VRAM for 16k-32k context on 16 GB GPUs.
Explore the active collections:
- MoE Standard (MiniPlus & NanoPlus): IsValorum/apex-i-miniplus-v21-standard-moe-6acfee77961c8c0cf5a083b5
- MoE Abliterated (MiniPlus & NanoPlus): IsValorum/apex-i-miniplus-v21-abliterated-moe-6acfee762fc4a8cf2298c55b
- 27B Dense Hybrids: IsValorum/val-apex-miniplus-standard-27b-6acfec54ca4ce4cf5cd5bdb6
- Apple Silicon (MLX): IsValorum/mlx-apex-abliterated-6abedb1f9d400e984efe4fc6
Full technical article and sizing breakdown: https://shortbench.vercel.app/
As the catalog approaches 400,000 total downloads across 69 releases, all models are now structured into dedicated official collections on Hugging Face.
Our goal remains consistent: eliminate the compromise between memory overflow and cognitive collapse on consumer hardware (especially 16 GB VRAM GPUs like the RTX 4080, RTX 4070 Ti Super, and RX 7800 XT).
Instead of uniform block quantization, all APEX builds use per-tensor mixed precision guided by 4-moment Importance Matrix statistics:
- MiniPlus Tier (approx. 14 to 14.8 GiB): Strict 0% 2-bit policy. All MoE routed experts locked at minimum IQ3_XXS with Q4_K/Q6_K attention. Designed for 24 GB setups or 16 GB setups with moderate context.
- NanoPlus Tier (approx. 11.5 to 12.3 GiB): Strategic use of IQ2_S/IQ2_XXS exclusively in core feedforward blocks, keeping routers and attention high-precision. Leaves ample VRAM for 16k-32k context on 16 GB GPUs.
Explore the active collections:
- MoE Standard (MiniPlus & NanoPlus): IsValorum/apex-i-miniplus-v21-standard-moe-6acfee77961c8c0cf5a083b5
- MoE Abliterated (MiniPlus & NanoPlus): IsValorum/apex-i-miniplus-v21-abliterated-moe-6acfee762fc4a8cf2298c55b
- 27B Dense Hybrids: IsValorum/val-apex-miniplus-standard-27b-6acfec54ca4ce4cf5cd5bdb6
- Apple Silicon (MLX): IsValorum/mlx-apex-abliterated-6abedb1f9d400e984efe4fc6
Full technical article and sizing breakdown: https://shortbench.vercel.app/
- 5 replies