flackzz/parakeet-ultra-mlx-q4-g32

Parakeet Ultra MLX Q4

MLX affine 4-bit conversion of Moondream Parakeet Ultra, using 32-element quantization groups for the acoustic encoder. The prediction network and joint network remain FP16. The three inference files total 418,297,763 bytes (418.3 MB, decimal).

This is the same multilingual Ultra model, with the same tokenizer and 25 European languages, including German and English. It is not Parakeet Redux or the original NVIDIA v3 checkpoint.

Runtime requirements

The encoder's linear layers and 48 pointwise Conv1d layers use packed UInt32 weights with FP16 affine scales and offsets. A runtime must support replacing the pointwise convolutions with quantized per-frame linear layers. The replacement squeezes [output, 1, input] to [output, input], preserving channel order and the convolution's output bias.

An MLX runtime supporting only quantized Linear/Embedding layers does not automatically support these pointwise-convolution weights. The weights should remain packed during inference; expanding the entire model to FP16 defeats the memory reduction. Read conversion.json for the exact format and source hashes.

Input is 16-kHz mono PCM. Audio capture, resampling, overlapping live windows, transcript merging and finalization belong to the application. This conversion does not turn Parakeet Ultra into a causal streaming model. The source MLX conversion already omitted Ultra's VAD head.

Files

File	Bytes	SHA-256
config.json	244,765	49ae51dcacc06ebc9f5cb57b12fbf3b73203d5645bc87e0ad12e3d8637167363
model.safetensors	418,006,226	621e8f0aa07b0fbcca5b6af70a54860c414182f05c75d164d69b761f26723e55
vocab.txt	46,772	3cde1409fd78783a79b29ed4d32da57c746993856f7c8263bcb905d2e5839db7

Conversion and validation

Source: beshkenadze/parakeet-ultra-mlx-fp16, revision 5acd3970b21853434afd56ed0c8f84a08a52cae8, a format conversion of moondream/parakeet-ultra. The source weights were verified by SHA-256 before conversion.

MLX affine quantization was applied to 265 encoder layers, including its 48 pointwise Conv1d layers. Subsampling convolutions, depthwise convolutions, normalization parameters and the prediction/joint network remain FP16. No fine-tuning or additional training data were used. The conversion uses the pinned MLX Swift implementation recorded in conversion.json.

Eight paired, locally synthesized German development clips covered negation, legal terminology, amounts, dates, an identifier and a German-English phrase. The principal German content was retained in this small check; formatting differences occurred. Both FP16 and Q4 made errors on the English phrase and omitted the spoken paragraph marker in a legal citation. This panel does not establish general German accuracy or legal-domain suitability.

One standalone Debug Mac run measured roughly 459 MB of active MLX allocations after loading and a 507 MB peak during these short clips. These values exclude total process footprint and an additional language model. Download size is not a guarantee of peak memory or iPhone latency. Physical iPhone/Catalyst memory and microphone validation remain application-specific.

License and attribution

CC BY 4.0, as the source model. See LICENSE.

• NVIDIA Corporation: original Parakeet TDT 0.6B v3.
• Moondream / M87 Labs: Parakeet Ultra, the post-trained model.
• beshkenadze: the source MLX FP16 format conversion.
• This repository: encoder Q4 group-32 quantization and corresponding configuration changes.

No endorsement by the original authors is implied. Model outputs require review when transcription errors could change the meaning.