Muse Mesh

We build Sanskrit-first language models, in the open: corpora, tokenizers and models trained from scratch, starting with Sanskrit. Every training run we do, finished or live, with its score and cost, is published at saansar.com.

Sansar: Sanskrit-first language models

Sansar is a family of causal language models trained from scratch, Sanskrit first. The models released so far are trained on Sanskrit text only, with a Sanskrit tokenizer of their own; none is a fine-tune of an English model. Other languages come later on the same recipe.

Scores are measured on frozen held-out Sanskrit sets: DCS gold sentences, prose, Vedic and out-of-domain texts. Held-out text is masked out of training. Every model card gives the per-set numbers, the training data and a version history.

Scored the same way, general base models need more bits per byte: Gemma 3 4B 0.6965, Qwen3-4B 0.7071, Llama 3.2 3B 0.7122, Sarvam-1 0.7465. sansar-700m beats all four on every held-out set with 704M parameters (Krutrim-2 12B, scored earlier in a pass that is not strictly comparable, is not in this list). A live demo is at saansar.com/demo.

from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("MuseMesh/sansar-700m", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("MuseMesh/sansar-700m", trust_remote_code=True)
ids = tok("संस्कृतं नाम दैवी वाक्", return_tensors="pt")
print(tok.decode(model.generate(**ids, max_new_tokens=60, do_sample=True, top_k=100)[0]))

Sansar OCR

sansar-ocr-2b reads printed Sanskrit (Devanagari) pages: a 2B vision-language model fine-tuned from Qwen3.5-2B on clean labels only. Median letter error on 54 held-out pages: 1.05%, against 1.69% for Google Cloud Vision and 9.03% for archive.org's OCR. It runs on a single consumer GPU.

English and Math models

Small models trained from scratch with the same recipe (Muon, rotary positions, QK-norm, squared ReLU) and a shared 32k tokenizer, released with their evaluation sets and the exact list of training documents. Collection.

• mume-tokenizer-32k: the shared 32k unigram tokenizer.
• mume-eval-suites: the frozen test sets behind every number (FineWeb val, WikiText-103, enwik8, text8, GSM8K, MATH, OpenWebMath val) with contamination-clean variants, plus training manifests.

Data and tokenizer

• Sansar Sanskrit Corpus: 252M words of openly licensed Sanskrit from 21 sources, in Devanagari. It is split into one config per licence (CC BY, CC BY-SA, CC0, ODC-By, Apache-2.0, MIT), and every record carries its source, licence and attribution. Text overlapping our evaluation sets is removed.
• sansar-sanskrit-tokenizer: an 8k unigram SentencePiece model over SLP1 transliteration with a Devanagari wrapper. 2.6 to 3.9 tokens per Sanskrit word with only 8k entries.

Licences

The corpus records keep their source licences. Model weights and the tokenizer are released under CC BY-NC 4.0 for non-commercial research use, and the modelling code under Apache-2.0. For other uses, contact us.

Links: muse-mesh.com · GitHub · kushal@muse-mesh.com