SmollerTTS

A 2.81M-parameter English TTS model that runs on a browser CPU. Built by Dun Li Chan.

SmollerTTS combines our custom acoustic model, trained from random initialization on LJSpeech, with the pretrained HiFi-GAN LJ_V2 vocoder. The release contains the accepted FP32 ONNX weights and the browser inference code used with them.

Live demo Β· Source repository

Model

Language / voices English / one voice
Acoustic model 1,882,807 parameters, trained by us
Vocoder 925,985 parameters, pretrained HiFi-GAN LJ_V2
Total 2,808,792 parameters
Audio 22,050 Hz, mono
Weights FP32 ONNX, about 11.34 MB across three graphs
Inference ONNX Runtime Web, WASM CPU, one thread

Parameter counts cover the acoustic model and waveform generator. Dictionaries, text-normalization rules and runtime binaries take additional disk space. This release uses full-precision weights.

Try it

Open the live demo to generate speech, adjust delivery and download a WAV. The voice runs on your device after its files load. No account or GPU is needed.

Run locally

Download the repository, then serve the files over localhost:

pip install huggingface_hub
hf download dunli5107/SmollerTTS --local-dir SmollerTTS
python -m http.server 8000 --directory SmollerTTS

Open http://localhost:8000/example.html, enter text and select Generate speech. The bundled example generates and plays a WAV entirely on the browser CPU. It includes all necessary runtime assets; no cloud inference API is used. Serve it over localhost or HTTPS rather than opening the HTML as a file.

This is a custom ONNX pipeline, not a Transformers pipeline() model. The live demo runs in the browser; this repository does not provide a Hugging Face inference endpoint.

Browser integration

worker.mjs loads the three graphs and pronunciation files, checks downloaded asset hashes and returns audio chunks. Keep the runtime files together in the same served directory. A minimal call is:

const worker = new Worker(new URL('./worker.mjs', import.meta.url), {type: 'module'});
worker.onmessage = ({data}) => {
  if (data.type === 'ready') {
    worker.postMessage({
      type: 'generate', id: 1, text: 'Hello, welcome to SmollerTTS.',
      controls: {speed: 1, pitch: 0, energy: 0},
      pronunciation: {splitCompounds: true, punctuation: true}
    });
  }
  if (data.type === 'chunk') {
    // data.wave: Float32Array, mono audio at 22,050 Hz.
  }
  if (data.type === 'error' || data.type === 'load-error') console.error(data.message);
};

example.html shows how to join the chunks, play them and save a WAV. Keep ONNX Runtime graph optimizations disabled, as configured in engine.mjs: alternate fusion settings changed output in this model's parity checks.

Architecture and training

SmollerTTS architecture: phoneme encoder, acoustic features and HiFi-GAN waveform decoder

Read each column from bottom to top. Explore the interactive diagram.

Text normalization and a pronunciation dictionary produce phoneme/stress tokens. The acoustic model uses 160-wide embeddings, four attention/convolution text blocks, duration and phone-prosody predictors, a length regulator, and six convolutional frame blocks. An 80-channel mel projection feeds HiFi-GAN V2. The decoder predicts pitch, voicing and energy as part of its conditioning.

The released acoustic run used 12,134 LJSpeech training recordings and 10,000 updates, with a fixed seed of 2718. Training conditioned the acoustic decoder on recorded prosody; inference uses its predictions. The pretrained vocoder was frozen. No synthetic teacher audio was used for this acoustic run. The complete system therefore includes pretrained weights; only our acoustic component was trained from scratch.

Browser preprocessing handles common times, dates, prices and units. Unknown words use pronunciation estimates; automatic compound handling is limited to four previously tested unfamiliar words. Delivery controls adjust duration, pitch and energy, but do not add speakers or learned emotional styles.

Evaluation

On 48 fixed English texts across six categories, all 48 SmollerTTS outputs completed. The same test also rendered both published Tiny Voca voices, with one fixed checkpoint and default controls per configuration.

Model / voice Parameters UTMOS ↑ (95% interval) Corpus WER ↓
SmollerTTS 2.81M 3.77 (3.68–3.86) 0.92%
Tiny Voca F2 4.54M 4.43 (4.41–4.46) 0.26%
Tiny Voca M2 4.54M 4.33 (4.28–4.37) 0.39%

All clips use the same pinned UTMOS22 strong judge and Whisper small.en recognizer at 16 kHz mono. Intervals use 10,000 stratified prompt-bootstrap replicates. They describe this text panel, not training-seed or speaker variability. UTMOS is an automatic quality predictor, not human MOS. Low WER does not imply natural speech; Tiny Voca scores higher on predicted quality.

WER uses a symmetric formatting pass for time punctuation and unit abbreviations, added after interim inspection. The original SmollerTTS WER was 1.71%; both versions and transcripts are retained. Tiny Voca's separate pronunciation model is excluded from its parameter count. Our browser runtime and its native PyTorch runtime are different, so no speed comparison is made.

See the fixed texts, per-clip results and protocol. Three sample clips below are the first three test sentences in order, not a selection by score.

Sample 2 Β· Sample 3

Limits

This is a small experimental English voice. It can sound muffled or robotic, and names, invented words and unusual text can be mispronounced. Punctuation controls are heuristic. There is no voice cloning, speaker selection or multilingual support. Performance depends on the browser and CPU; real-time generation is not guaranteed on every device. The evaluation is a small automated panel, not a human listening study.

Files and attribution

  • encoder.onnx, frames.onnx, vocoder.onnx: inference weights.
  • model.json: architecture, parameter counts and weight provenance.
  • Pronunciation vocabulary, dictionary and bundled frontend: frontend.json, en.lex and the frontend module loaded by worker.mjs.
  • worker.mjs, engine.mjs, helper modules and ort/: browser inference.
  • assets.json, SHA256SUMS: file identities for verification.
  • example.html: minimal local example.

Our code and model release use the MIT license. Third-party notices accompany their components: HiFi-GAN and ONNX Runtime (MIT), phonemize and to-words (MIT), and the adapted MFA pronunciation dictionary (CC BY 4.0). See THIRD_PARTY.md and the included license files.

The acoustic model was trained on LJSpeech. The waveform generator uses the authors' published HiFi-GAN LJ_V2 weights, not a vocoder trained by this project. FastSpeech 2 informed the duration/prosody design; UTMOS provides the automatic quality estimator. These architectural references are credited, not claimed as new algorithms.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Papers for dunli5107/SmollerTTS