Duckets/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit

NVIDIA Nemotron 3.5 Super VL 120B-A12B — MLX 4-bit

An MLX 4-bit conversion of NVIDIA's Nemotron 3.5 Super VL 120B-A12B (BF16), built so the model runs natively on Apple Silicon with mlx-vlm and in LM Studio.

This is the full omni model, vision included — the Nemotron-H hybrid language model plus its C-RADIO vision tower, converted from the BF16 source (not from a GGUF). The source checkpoint has no audio tower (sound_config is null), and neither does this one.

Conversion details

Source	nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16
Quantization	MLX affine 4-bit, group size 64, ~4.57 bits/weight
Size on disk	64.7 GiB (69.4 GB), 14 safetensors shards
Total / active parameters	120B / 12B
Architecture	Nemotron-H hybrid (interleaved Mamba-2, attention, and MoE layers; 512 routed experts, top-22) + C-RADIO vision encoder
Context length	1,048,576 tokens
Inputs	Text and images (video frames are supported upstream; verified here for text + image)
Output	Text

Config fixes applied versus a raw conversion:

• Removed the redundant _downsample_factor key from the processor configs (the value is derived from downsample_ratio; the literal key breaks NVIDIA's own processor code, which exposes it as a read-only property).
• Normalized time_step_limit's Infinity sentinel to a plain number (1e30) so strict JSON parsers accept config.json.
• Added a top-level max_position_embeddings (1048576) so front-ends report the true context length instead of a 4096 default.
• The shipped chat_template.jinja is null-hardened at 7 sites so strict jinja renderers (LM Studio's) don't fail on tool schemas containing null values. Output is unchanged for non-null input.

Verified

• Generates in mlx-vlm — text and image description both tested.
• Loads and serves in LM Studio at the full 1,048,576 context, with text and vision answers verified through LM Studio's own API.
• Loads on a 128 GB Apple Silicon Mac in about 10 seconds. Plan on a 128 GB machine; the weights alone are ~65 GiB resident.

Usage

With mlx-vlm:

pip install -U mlx-vlm

python -m mlx_vlm.generate \
  --model Duckets/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-MLX-4bit \
  --prompt "Describe this image." \
  --image /path/to/image.png

With LM Studio: download the repo into your LM Studio models folder and load it from the MLX models list. Two things to know:

• Load with KV Cache Quantization OFF — with it on, LM Studio's mlx-vlm batched-vision path rejects the load / ignores the context-length setting.
• This architecture is newer than some LM Studio backends: at upload time, a stock backend was missing the nemotron_h_omni mlx-vlm model package and two small processor dependencies, so a plain load could fail on engine grounds alone. That is an engine-side gap, not a problem with these weights — the same files run as-is under mlx-vlm. Newer LM Studio builds may already include what's needed.

Reasoning is configurable through the chat template (enable_thinking), as in the upstream release.

Limitations

• 4-bit quantization can differ slightly from the BF16 original on borderline outputs; use the upstream BF16 release for full-precision evaluation.
• Vision was verified with images. Video input follows the upstream design but was not part of this port's verification.
• No audio input or output — the source model has no audio tower.

License

Use is governed by the upstream license: the OpenMDW License Agreement, version 1.1. This is a derivative conversion of NVIDIA's model; all credit for the model itself, its training data, and its capabilities belongs to NVIDIA. See the upstream model card for the full description, benchmarks, and training details.

---

Conversion and packaging by Juno for Duckets, 2026-10-15.