spokedotso/cohere-transcribe-03-2026-mlx-4bit

A 4-bit MLX quantization of CohereLabs/cohere-transcribe-03-2026, hosted by Spoke for on-device speech-to-text on Apple Silicon.

Derived from the native mlx-speech conversion (mlx-community/cohere-transcribe-03-2026-mlx-8bit, maintained by appautomaton) by dequantizing the int8 weights and re-quantizing to 4-bit (affine, group size 64) with mlx-speech's own quantizer. Loads via the mlx-speech library (Python 3.13+), not stock mlx-audio.

Why this exists

On Spoke's internal LibriSpeech test-clean benchmark (250 samples), this 4-bit build measured 1.28% WER at 0.05x real-time factor with a 1.89 GB peak memory footprint, beating the previously shipped 4-bit Whisper turbo (2.66% WER) on accuracy and speed while staying under a 2 GB memory budget.

Usage

import numpy as np, soundfile as sf
from mlx_speech.generation import CohereAsrModel

audio, sr = sf.read("input.wav", dtype="float32", always_2d=False)
if audio.ndim > 1:
    audio = audio.mean(axis=1)
# resample to 16 kHz first if sr != 16000

model = CohereAsrModel.from_path("path/to/this/repo")
print(model.transcribe(audio, sample_rate=16000, language="en").text)