mimech011/nllb-200-kabyle-1.3B

🧠 NLLB-200 Kabyle 1.3B (Bi-directional Translator)

mimech011/nllb-200-kabyle-1.3B is a state-of-the-art, 1.3 Billion parameter sequence-to-sequence translation model natively optimized for the Kabyle language (Taqbaylit) [1.1.2, 1.2.1, 2.4.4].

It is fine-tuned from Meta's nllb-200-distilled-1.3B [1.1.7] using a highly balanced, multi-task dataset consisting of 677,528 parallel translation directions [1.1.2, 1.2.1, 2.4.4].

Unlike standard bilingual models, this is a 4-way bi-directional translator [2.4.4]. A single 1.3B model handles translations in all of these directions simultaneously:

• English $\rightarrow$ Kabyle (eng_Latn $\rightarrow$ kab_Latn) [1.1.2]
• Kabyle $\rightarrow$ English (kab_Latn $\rightarrow$ eng_Latn)
• French $\rightarrow$ Kabyle (fra_Latn $\rightarrow$ kab_Latn) [1.2.1]
• Kabyle $\rightarrow$ French (kab_Latn $\rightarrow$ fra_Latn)

---

📈 Model Performance & Advantages

• High Vocabulary Precision: Thanks to the 1.3B parameter capacity, the model does not suffer from "capacity bottlenecking"
• Zero Hallucination Risk: As an Encoder-Decoder model, it is mathematically constrained to only output direct translations [2.3.4]. It will never output generic chat loops or filler words.
• Low-Resource Generalization: It leverages Meta's multilingual pre-training on related Amazigh dialects (such as Tachelhit/shi_Latn), resulting in highly fluent, context-aware grammatical outputs in Kabyle [2.4.4].

---

🛠 Training Procedure

The model was fine-tuned on an NVIDIA A100-40GB GPU using the following hyperparameters:

• Base Model: facebook/nllb-200-distilled-1.3B [1.1.7]
• Dataset Size: 677,528 unified bilingual directions [1.1.2, 1.2.1]
• Epochs: 1 full epoch [1.1.2, 1.2.1]
• Optimizer: adamw_bnb_8bit (8-bit AdamW)
• Learning Rate: 5e-5 (with linear decay)
• Active Batch Size: 32 (with gradient_checkpointing=True enabled for VRAM stability) [2.1]
• Dataset Sources: English-Kabyle Tatoeba Parallel Corpus [1.1.2] and French-Kabyle Sifal/Tatoeba Corpus [1.2.1].
• Final Evaluation Loss: 1.689 (Highly stable convergence)

---

🚀 Quick Start (Inference in Python)

To run this model on your local GPU (such as an RTX 2070 / 3060), you can load it in FP16 precision [2.1] to reduce VRAM consumption from 5.5 GB down to only ~2.7 GB [2.1].

import torch
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer

model_name = "mimech011/nllb-200-kabyle-1.3B"

# Load in FP16 to keep VRAM usage low (~2.7 GB)
model = AutoModelForSeq2SeqLM.from_pretrained(
    model_name, 
    torch_dtype=torch.float16
).to("cuda")

tokenizer = AutoTokenizer.from_pretrained(model_name)

def translate(text, src_lang, tgt_lang):
    tokenizer.src_lang = src_lang
    inputs = tokenizer(text, return_tensors="pt").to("cuda")
    
    # Force the decoder to start with the correct target language token
    forced_bos_token_id = tokenizer.lang_code_to_id[tgt_lang]
    
    translated_tokens = model.generate(
        **inputs, 
        forced_bos_token_id=forced_bos_token_id, 
        max_length=128,
        num_beams=4 # Standard beam search for high-quality translations
    )
    
    output = tokenizer.batch_decode(translated_tokens, skip_special_tokens=True)[0]
    return output

# Example 1: English -> Kabyle
text_en = "Write a short explanation of how a small Kabyle family shop can use a smartphone to get more customers."
print("[KB]:", translate(text_en, "eng_Latn", "kab_Latn"))

# Example 2: French -> Kabyle
text_fr = "Où est le médecin le plus proche ?"
print("[KB]:", translate(text_fr, "fra_Latn", "kab_Latn"))