AfriSpeech/africa-whisper-checkpoints

Africa-Whisper: Universal Multilingual Speech Recognition for 314 African Languages

Africa-Whisper is an open-source, community-driven speech recognition foundation model adapted from OpenAI's Whisper Large-v3-Turbo (809M parameters) across 314 African languages and dialects.

By conditioning Whisper on the English transcription prompt (<|startoftranscript|> <|en|> <|transcribe|> <|notimestamps|>), Africa-Whisper leverages Whisper's vast multilingual subword priors to transcribe spoken audio across 314 indigenous African languages directly into a regularized Latin orthography with preserved word boundaries (single spaces) and stripped punctuation, generated using Africa-G2P.

• GitHub Repository: https://github.com/AfriSpeech/africa-whisper
• Pre-Training Corpus: AfriSpeech/afri2universal (314 languages, ~950.6 hours)
• License: Creative Commons Attribution-NonCommercial 4.0 (CC-BY-NC-4.0)

---

Final Release Metrics (Checkpoint-76000 / Epoch 16.62)

We selected Step 76,000 (Epoch 16.62) as the official foundation release checkpoint because it delivers the optimal balance between phonetic character fidelity and whole-word recognition accuracy across all 314 held-out evaluation sets:

Metric	Score across 314 African Languages
Word Error Rate (WER)	45.74% (All-time best in the entire training run)
Character Error Rate (CER)	20.50% (Near all-time record of 20.09%)
Evaluation Loss (Cross-Entropy)	0.7528 (60.7% total loss reduction from 1.9156 baseline)
Training Steps Completed	76,000 steps (Effective batch size 64 = 4.86M sample exposures)
Full Passes over Corpus	16.62 full epochs

---

A Call for Community Fine-Tuning 🌍

Africa-Whisper is designed as a collaborative community project. While this foundation checkpoint has learned robust cross-lingual acoustic representations across 314 African languages, speech recognition accuracy on specific dialects, accents, and domains can be significantly enhanced by fine-tuning on additional speech corpora.

We warmly invite researchers, developers, and language communities across Africa to take this foundation checkpoint and fine-tune it further!

Datasets Trained On

Dataset	Languages	Total Duration	Utterances	Families Represented
Afri2Universal	314	~950.6 hours	292,591	Niger-Congo, Afroasiatic, Nilo-Saharan, Khoisan

Suggested Datasets for Continued Fine-Tuning

We encourage the community to fine-tune Africa-Whisper on additional African speech corpora:

1. Waxal Corpus: High-quality speech for West African languages (Wolof, Pulaar/Fulfulde, Mandinka, Jola, Serer).
2. OmniASR African Subsets: Multi-dialect African audio benchmarks.
3. Mozilla Common Voice (Africa): Crowdsourced voice recordings (Kinyarwanda, Luganda, Swahili, Hausa, Yoruba, etc.).
4. Google FLEURS: 102-language benchmark with African language splits.
5. ALFFA (African Languages in the Field): Field recordings for Amharic, Oromo, Tigrinya, and Wolof.
6. Local Broadcast & Community Radio Archives: Community radio and news broadcasts.

---

Standardized Data Preparation (Africa-G2P)

To ensure that fine-tuning on new datasets maintains token alignment and cross-lingual acoustic transfer, all new transcripts must be pre-processed using the identical orthographic normalization standard:

1. Convert text to universal 26-character lowercase Latin (a-z).
2. Preserve single spaces between words ( ) to enable true word boundary decoding.
3. Strip all punctuation, tone accents, and non-Latin glyphs.
4. Audio format: 16,000 Hz mono FLAC or WAV.

We provide the standalone script prepare_dataset.py in this repository:

pip install git+https://github.com/AfriSpeech/africa-g2p.git soundfile librosa datasets

from prepare_dataset import universalize_text

# Example: Twi
clean = universalize_text("Onyankopɔn, me din de Kwame! Ɔyɛ kɛse.", lang="twi")
print(clean)
# -> 'onyankopon me din de kwame oye kese'

# Example: Amharic (Ge'ez script)
clean = universalize_text("ሰላም፣ ስሜ አማን ነው።", lang="amh")
print(clean)
# -> 'salaame semee amaane nawe'

---

Quickstart: How to Run Inference

You can load and run Africa-Whisper with 3 lines of code using Hugging Face transformers and peft:

pip install torch transformers peft soundfile librosa

import torch
from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq
from peft import PeftModel

device = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if torch.cuda.is_available() else torch.float32

# 1. Load base Whisper Large-v3-Turbo
base_model_id = "openai/whisper-large-v3-turbo"
processor = AutoProcessor.from_pretrained(base_model_id)
model = AutoModelForSpeechSeq2Seq.from_pretrained(
    base_model_id,
    torch_dtype=dtype,
    low_cpu_mem_usage=True
).to(device)

# 2. Attach Africa-Whisper LoRA adapter directly from root
model = PeftModel.from_pretrained(model, "AfriSpeech/africa-whisper-checkpoints")
model.eval()

# 3. Transcribe speech from any of the 314 African languages
def transcribe_audio(audio_path):
    import librosa
    audio, _ = librosa.load(audio_path, sr=16000)
    inputs = processor(audio, sampling_rate=16000, return_tensors="pt").input_features.to(device, dtype=dtype)
    forced_decoder_ids = processor.get_decoder_prompt_ids(language="en", task="transcribe", no_timestamps=True)
    
    with torch.no_grad():
        predicted_ids = model.generate(inputs, forced_decoder_ids=forced_decoder_ids, max_length=448)
        
    return processor.batch_decode(predicted_ids, skip_special_tokens=True)[0].strip()

transcript = transcribe_audio("sample_african_audio.wav")
print("Universal Transcription:", transcript)

Alternatively, use the included CLI tool:

python transcribe.py --audio sample.wav

---

How to Continue Fine-Tuning on Your Own Dataset

Use the included finetune_whisper.py script:

python finetune_whisper.py \
    --adapter_model_id "AfriSpeech/africa-whisper-checkpoints" \
    --dataset_name "google/fleurs" \
    --subset "yo_ng" \
    --lang_code "yor" \
    --output_dir "./output-africa-whisper-fleurs" \
    --batch_size 16 \
    --learning_rate 5e-5 \
    --num_epochs 5

---

Repository Structure

├── adapter_config.json          # LoRA configuration (r=32, alpha=64, q/v projections)
├── adapter_model.safetensors    # Final released LoRA weights (Step 76,000)
├── optimizer.pt                 # AdamW optimizer state for exact resumption
├── scheduler.pt                 # Cosine/Linear learning rate scheduler state
├── trainer_state.json           # Complete evaluation history across all 98k steps
├── training_args.bin            # Full training arguments
├── prepare_dataset.py           # Standardized Africa-G2P dataset preparation script
├── finetune_whisper.py          # Script for continuing fine-tuning on new datasets
├── transcribe.py                # Standalone inference & CLI transcription script
├── best/                        # Archive of best WER checkpoint (Step 76,000)
├── best_cer/                    # Archive of best CER checkpoint (Step 76,000)
└── last/                        # Archive of latest training step

---

Citation & Acknowledgements

@model{africa_whisper_2026,
  author    = {AfriSpeech},
  title     = {Africa-Whisper: Universal Multilingual African Speech Recognition (314 Languages)},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/AfriSpeech/africa-whisper-checkpoints}
}

@software{africa_g2p_2024,
  author       = {AfriSpeech},
  title        = {Africa-G2P: Grapheme-to-Phoneme and Universal Orthographic Converter for African Languages},
  year         = {2024},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/AfriSpeech/africa-g2p}}
}

@dataset{afri2universal_2026,
  author    = {AfriSpeech},
  title     = {Afri2Universal: Universalized Multilingual African Speech Corpus Across 314 Languages},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/datasets/AfriSpeech/afri2universal}
}