thelogicalgate/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX

NVIDIA Nemotron 3.5 Bolt 74B-A9B (MLX)

MLX safetensors conversion of NVIDIA Nemotron 3.5 Bolt 74B-A9B. The language model and RADIO vision tensors are in the repository root. The LiLiCorr speculative drafter is in drafter/.

This is a derived weight conversion. Use is governed by the OpenMDW License, version 1.1.

Sources

• Language model and vision encoder: nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-BF16
• LiLiCorr drafter: nvidia/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB-LiLiCorr (NVFP4 source, stored in this repository as BF16)

Architecture

Parameters	74B total, 9B active
Layers	88 (Mamba-2, attention, and latent MoE)
Hidden size	4096
Attention	32 query heads, 2 key/value heads, head dimension 128
Experts	up to 496 routed experts, top-9, one shared expert
Latent size	1024
Vocabulary	131072
Context length	262144
Vision	C-RADIO v4-H, 32 layers, hidden 1280, 16 heads, pixel-shuffle projector (mlp1) into 4096
Image token	<image>, id 18

The MoE block routes on the full hidden state, runs the routed experts in the 1024-wide latent, and projects back. The shared expert stays on the original hidden state.

Vision tensors keep the original RADIO names (fused QKV, mlp.fc1 / mlp.fc2) plus mlp1. The checkpoint also contains patch_generator.video_embedder for temporal frame pairs. Image and video serving is supported by the native RADIO path in mlx-serve PR #870.

Quantization

Affine quantization, group size 64. Weights are packed uint32 values with bfloat16 scales and biases. config.json records bits: 4 and group_size: 64. The packed width depends on the tensor:

Tensors	Bits
Routed experts (switch_mlp), shared experts, Mamba in_proj and out_proj	4
Token embeddings, lm_head, attention Q/K/V/O, latent projections	8
Norms, Mamba conv, A_log, D, dt_bias, router gate	BF16
RADIO encoder and mlp1	BF16

Packing is along the input axis: 8 values per uint32 at 4 bits, 4 values per uint32 at 8 bits.

Drafter

drafter/config.json and drafter/model.safetensors.

• Trunk: 5 sliding-window attention layers, hidden 4096, 32 query heads, 2 key/value heads, window 1024
• LiLiCorr head: hidden 1024, 2 layers, 8 heads, candidate top-k 8, block size 8
• Target layer ids: 1, 18, 35, 51, 68, 85
• Stored dtype: BF16, plus one FP32 attention bias of shape [8, 56, 56]
• The drafter config has no quantization_config

A loader that keeps each layer's output should feed LiLiCorr the previous layer's output. The declared target id is that output index plus one, because the head was trained on the layer input.

Layout

• model-00001-of-00050.safetensors through model-00050-of-00050.safetensors, with model.safetensors.index.json
• vision-00000.safetensors, vision-00001.safetensors, vision-00002.safetensors (vision_model.* and mlp1.*)
• config.json is the serving config: it includes the native vision_config.model_type: "radio", downsample ratio, temporal patch size and video pruning rate. config.hf.json retains the upstream config for provenance; serving does not require reading it
• Chat template, tokenizer, and processor files are carried from the BF16 release
• bias.md, privacy.md, explainability.md, and safety.md are the upstream NVIDIA policy notes

Serving

Use a build containing mlx-serve PR #870, which reads the native RADIO metadata in config.json, the indexed vision shards, and the drafter/ sidecar. No manual config edits are required.

hf download thelogicalgate/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX --local-dir "<target>"
mlx-serve --model "<target>" --serve

The target's drafter/ subdirectory is discovered automatically. Still images use the <image> token. Video requests supply client-decoded frames through video_url.frames; the engine processes the supplied sequence through the trained temporal embedder. Container decoding and frame sampling belong to the client.

License

OpenMDW-1.1.