Mia-AiLab/MiniMax-M3.1-Flash-EXL3-EAQ-v1-3.5-bpw

MiniMax-M3.1-Flash · EXL3 · EAQ v1 · 3.5 bpw

EAQ v1 (EXL3 format) quantization of MiniMaxAI/MiniMax-M3.1-Flash, 60 layers, 128 routed experts with top-4 routing, 3.55 bits per weight on average (target 3.5) and a 6-bit LM head. The weights are 179.4 GiB (192.6 GB) in 25 shards.

Status: private build, quality measured, not yet run end to end. The pack has passed structure, finiteness and hash checks, and it has been measured against the original model by teacher-forced scoring (below). Loading it in an inference engine and generating text are NOT TESTED. No engine we run today has verified support for this architecture, and the pack does not fit on a single 96 GB GPU.

At a glance

Base model	MiniMaxAI/MiniMax-M3.1-Flash (official MXFP8 checkpoint)
Format	EXL3 (trellis / suh / svh), codebook mul1, out_scales=always, exllamav3 1.6.1
Average bits per weight	3.55 as recorded in quantization_config.json (target 3.5)
LM head	6 bit
Size	quantized text model 179.4 GiB (192,592,265,620 bytes) in 25 shards, 89,832 tensors; plus 2.3 GB dspark and 1.7 GB vision, about 196.7 GB in the repo
Layers	60 (3 dense-MLP layers, then 57 MoE layers with 128 routed experts, top-4, plus a shared expert)
Declared context	1,048,576 tokens (from the base config; long context is NOT TESTED here)
Extras (verbatim)	dspark/: the base model's speculative-decoding draft (2.3 GB, 5 layers, block size 7), and model-vision-00001-of-00001.safetensors: the vision tower and projector (523 tensors, 1.7 GB). Both are copied byte-for-byte from the base model, not requantized. The base model has no MTP head; speculative decoding is the dspark draft

What is quantized, and what is not

Component	Stored as
Attention q / k / v / o	6 bit
Dense MLP (layers 0-2)	5 bit
Shared expert (layers 3-59)	5 bit
Routed experts, layers 3-16	4 bit
Routed experts, layers 17-45	3 bit (layer 17 mixed 3 / 4)
Routed experts, layers 46-59	4 bit
LM head	6 bit
Sparse-attention indexer (index_q_proj, index_k_proj)	FP16, not quantized
MoE router (gate.weight, e_score_correction_bias)	FP32, not quantized
Embeddings, norms	BF16, not quantized

The indexer chooses which tokens attention reads, and the router chooses which experts run. Small errors there change what the model looks at and which weights it uses, so both are kept at source precision in every layer. This was checked on the finished pack: all 60 layers.

Quality vs the original model

Method: teacher-forced next-token scoring. The original checkpoint (its MXFP8 / NVFP4 blocks dequantized) and this pack both read the same token ids; for every assistant token we compute KL(original || quantized) over the full vocabulary, and whether the two models pick the same top-1 token. Lower KL is better; 0 means identical distributions. Intervals are 95% bootstrap CIs over conversations.

Evaluation set: 92 conversations, 123,440 tokens, 58,920 scored assistant tokens, rendered with MiniMax's own chat template, rows cut at 2,048 tokens (the port's limit; both sides see identical ids). The set is spread over agentic, tool calling, code, chat, multilingual and thinking (reasoning) variants.

Pooled result (MEASURED): mean KL 0.0566 nats [0.0286, 0.0946], median 0.0025, top-1 agreement 94.3%. The mean sits well above the median (0.0025) because a small share of tokens diverges strongly (KL > 1 on 0.78% of tokens, KL > 0.1 on 7.3%). Teacher and student cross-entropy on the reference text is nearly identical (NLL 1.4254 vs 1.4236).

Workload group	Conversations	Mean KL (95% CI)	Top-1 agreement (95% CI)
Tool calling + agentic (incl. thinking)	35	0.0127 [0.0092, 0.0172]	97.1% [96.4%, 97.7%]
Code (incl. thinking code)	21	0.0278 [0.0129, 0.0587]	96.1% [95.7%, 96.7%]
Chat, multilingual, thinking chat, thinking STEM	36	0.0959 [0.0418, 0.1668]	91.8% [89.6%, 93.8%]
All scored tokens	92	0.0566 [0.0286, 0.0946]	94.3% [93.1%, 95.4%]

Reading it honestly:

• Strong: tool calling and agentic work. Mean KL about 0.013 and top-1 agreement 97.1% across 35 conversations, and the median token is essentially unchanged. This is the workload this pack is best suited for.
• Good, with a tail: code. Median KL is near zero and top-1 agreement is 96.1-96.8%, but the plain Code stratum has a heavy tail (40 tokens above KL 1), which is why its mean (0.060) is higher than thinking-code (0.014). Expect usually identical output with occasional large divergences.
• Weak: chat and, above all, multilingual. Multilingual mean KL is 0.223 and top-1 is 88.2%; chat is 0.121 and 89.1%. Those two strata hold 404 of the 450 tokens with KL above 1 that the per-stratum tally lists. If your use is conversation in many languages, this 3.5-bit build is not the right choice.
• Small samples. The thinking strata have 4-8 conversations each and the intervals are wide. Treat per-stratum numbers as indicative.

All strata

Stratum	Conversations	Mean KL (95% CI)	Top-1 agreement (95% CI)
Agentic	12	0.0092 [0.0059, 0.0134]	97.8% [97.1%, 98.4%]
Thinking agentic	6	0.0096 [0.0066, 0.0140]	97.5% [97.2%, 98.0%]
Tool calling	12	0.0152 [0.0086, 0.0259]	97.1% [96.0%, 97.7%]
Thinking tool calling	5	0.0189 [0.0085, 0.0287]	95.6% [93.9%, 97.6%]
Code	13	0.0604 [0.0117, 0.1285]	96.8% [95.9%, 97.9%]
Thinking code	8	0.0139 [0.0126, 0.0152]	95.9% [95.5%, 96.3%]
Thinking STEM	8	0.0177 [0.0144, 0.0213]	95.5% [94.9%, 96.2%]
Thinking chat	4	0.0297 [0.0277, 0.0323]	93.3% [91.8%, 94.3%]
Chat	10	0.1210 [0.0419, 0.2031]	89.1% [85.5%, 93.5%]
Multilingual	14	0.2225 [0.0634, 0.3737]	88.2% [83.8%, 92.7%]

Where the difference comes from

The first layers match the original almost exactly; the difference builds up through the stack. From about layer 25 on, the router picks the same top-4 experts as the original on only 50-70% of tokens, and the hidden state sits roughly 12-24% (relative L2) away from the original state. The model compensates: next-token distributions stay close on most text, as the numbers above show. We report this because it explains the heavy tail, and because it is what a deeper mix of bits (more layers at 4 bit) would address.

What was NOT tested

• Loading and generation. No end-to-end run in any inference engine. A generation sanity check was not possible on the available hardware (180 GB pack, 96 GB GPU) and no supported engine for this architecture was available.
• Long context. Everything above is at most 2,048 tokens. Behaviour past that, including the sparse-attention indexer at long range, is NOT TESTED.
• 8 conversations were dropped from the evaluation because the system and tool prompt alone exceeds 2,048 tokens. They were all tool / agentic conversations, so long agentic prompts are NOT TESTED.
• Vision and speculative decoding. The vision tower and the dspark draft are included verbatim but were never run here. The draft reads the target's hidden states at layers 3, 17, 31, 45 and 59. In this pack those states sit about 17-23% (relative L2) away from the original at layers 31, 45 and 59, so draft acceptance may be lower than with the original model. Acceptance rate and speedup are NOT TESTED.
• Other quantizations for comparison. None exist for this base model, so there is no like-for-like competitor in these numbers.
• Downstream benchmarks (task accuracy, coding pass rates, tool-call success rates) were not run. KL and top-1 agreement measure closeness to the original, not task quality.

Files

• model-0000N-of-00025.safetensors, model.safetensors.index.json: the quantized weights.
• dspark/: the speculative-decoding draft model from the base model (config, tokenizer files, model-dspark-00001.safetensors), unchanged.
• model-vision-00001-of-00001.safetensors: vision tower and projector tensors from the base model, unchanged, listed in model.safetensors.index.json.
• config.json, quantization_config.json, generation_config.json, tokenizer files, chat_template.jinja and the base model's configuration_minimax_m3_vl.py, processing_minimax.py, image_processor.py, video_processor.py for compatibility with the base model's loader.

License and credits

This repo contains a quantized derivative of MiniMax-M3.1-Flash. It does not change the base model's terms: follow the license of MiniMaxAI/MiniMax-M3.1-Flash. Quantization format and kernels are from exllamav3.