Ordicio on-device translator

Small translation models for the 16 languages of the Ordicio apps, which translate attribute values and data sheets of collectors' items on the device. There is one model per direction between English and each language; other pairs go through English. A language detector of 2.2 MB (0.75 MB to download) tells which model to use.

Languages	en, de, fr, es, it, nl, pt (Brazil), pl, sv, da, nb, fi, ja, ko, zh-Hans, zh-Hant
Models	30 directions (en↔x), transformer encoder + SSRU decoder, int8 (bergamot-translator intgemm8 with alphas)
Download	about 23 MB per direction, 13 MB for sv, da, nb and 34 MB for ja, ko, zh (gzip compressed); the detector 0.75 MB
Speed	about 18 ms per value in the WebAssembly engine on one core
Base	the Firefox Translations students of each direction
Training	fine-tuned on collecting texts translated by Gemma 4 31B, plus a quantization-aware epoch

Files

manifest.json lists every file with its sha256 and size (of the stored, gzip-compressed file) and the unpacked size:

{
 "id": "ordicio-translator-v1",
 "languages": ["en", "de", "..."],
 "pivot": "en",
 "engine": {"beam-size": "1", "gemm-precision": "int8shiftAlphaAll", "...": "..."},
 "pairs": {
  "en-de": {"from": "en", "to": "de",
            "model": {"path": "v1/en-de/model.bin.gz", "sha256": "…", "size": 0, "gzip": true, "unpackedSize": 0},
            "srcVocab": {"path": "v1/en-de/vocab.spm.gz", "...": "..."}, "trgVocab": {"...": "..."}}
 },
 "lid": {"model": {"path": "v1/lid.bin.gz", "...": "..."}, "labels": ["en", "...", "zxx"], "buckets": 131072,
         "ngrams": [1, 4], "scale": 0.0, "bias": [], "maxChars": 1000, "hash": "fnv1a32-utf8"}
}

• Paths are relative to the manifest. srcVocab and trgVocab name the same file when the direction has one vocabulary.
• engine holds the bergamot-translator options the models were evaluated with. They run without a lexical shortlist (it cost quality on short values).
• source/v1/<src>-<tgt>.npz are the float Marian weights of each model (the form to change them in); the apps do not download them.

Language detector

A linear model over hashed character n-grams with 17 labels: the 16 languages and zxx (names, codes, model numbers: nothing to translate). lid.bin is int8, buckets × labels (the weights of one bucket together).

1. Text → lower case → every digit 0-9 to 0 → runs of whitespace (space, tab, line feed, carriage return, form feed, vertical tab, U+00A0, U+1680, U+2000–U+200A, U+2028, U+2029, U+202F, U+205F, U+3000) to one space → spaces trimmed → the first maxChars code points. No Unicode normalization.
2. Each word w (split on spaces) is padded to " " + w + " "; every n-gram of 1 to 4 code points of the padded word except a single space counts once.
3. The bucket of an n-gram is the 32-bit FNV-1a hash of its UTF-8 bytes modulo buckets. The feature vector holds the count per bucket, L2 normalized.
4. Score of label i = bias[i] + scale × Σ feature × weight[bucket][i]; softmax gives probabilities.

Accuracy (16 languages)	this detector	Lingua	fastText lid.176
short values (held out, 6,877)	0.89	0.83	0.74
sentences (WMT / NTREX held out, 2,649)	1.00	0.99	0.98

Short values are often ambiguous (Danish and Norwegian, Spanish and Portuguese, Chinese characters in Japanese). On them, 78 % of the predictions reach a probability of 0.9, and 98.7 % of those are right.

Quality

chrF++ in the bergamot WebAssembly engine without shortlist, against the shipped Firefox model of the same direction. Test: 150 English values of a collecting catalog (examples, list values, member notes) with reference translations.

en →	de	fr	es	pl	fi	ja	ko	zh-Hans
Firefox	69.1	67.5	67.8	60.0	60.7	32.5	42.5	25.1
this	70.7	70.9	69.2	64.4	65.1	40.6	46.1	36.3

Values that every reference keeps unchanged (names, titles, codes) stay unchanged twice as often (9 to 16 of 19 instead of 1 to 11).

→ en (the references as source)	de	fr	es	pl	fi	ja	ko	zh-Hans
Firefox	75.0	74.4	73.6	69.3	66.0	58.2	57.9	55.0
this	76.8	74.7	75.4	69.4	67.4	59.2	61.9	59.0

it, nl, pt, sv, da, nb and zh-Hant have no reference set. On 300 held-out teacher translations they gain 4 to 32 chrF++ over the Firefox models from English and 1 to 9 into English.

Training

1. 10,900 English collecting texts (attribute values of a classification catalog, member notes) translated by Gemma 4 31B (4-bit) into the 15 languages.
2. Each Firefox student (float weights from Mozilla's published training runs) fine-tuned on these pairs (both directions; for x → en the teacher's translation is the source) mixed with WMT and NTREX-128 sentences: 4 epochs, learning rate 2e-5, then one quantization-aware epoch (8 bit), so the int8 model scores like the float one.
3. For Korean, teacher pairs that copy a text of three or more words unchanged are left out (the model learned to leave whole sentences in English).

License

The models are derived from the Firefox Translations models and are released under the Mozilla Public License 2.0 like them. The teacher, Gemma 4, is licensed under Apache 2.0. Fine-tuning data also includes NTREX-128 (CC BY-SA 4.0) and WMT test sets.