mazun/rekord

Okarina note-transcription models (ONNX) for Re:Kord

The ONNX models Re:Kord uses for real-time polyphonic note transcription of music with vocals and drums.

These are not the paper authors' weights. Okarina is an unofficial reimplementation and derivative of Harmonica (instrument-agnostic note transcription, arXiv:2609.04640), trained from scratch and exported to ONNX (fp32) in the separate harmonica-repro project. It comes in three sizes, S, M and L. Each separates the notes into three parts: vocal, bass and other (everything else; drums are not transcribed).

Files

File	What it is	Output
cqt156.onnx	Constant-Q transform front end, shared by all sizes (no parameters: fixed Hann-windowed complex constant-Q kernels)
okarina_s.onnx	Okarina S (0.7 MB)	[B, 3, T, 88]
okarina_m.onnx	Okarina M (2.9 MB)	[B, 3, T, 88]
okarina_l.onnx	Okarina L (13.1 MB, the large architecture, 3.22M parameters)	[B, 3, T, 88]

The front end takes 109,869 samples of 16 kHz mono audio and returns a 156-frame × 264-bin CQT (5 s at 32 ms per frame). A note model takes cqt_mag [B, T, 264] and returns onset, sustain and offset probabilities for 88 pitches (MIDI 21–108) per frame, each [B, 3, T, 88]; the second axis is the part, in the order vocal, bass, other.

The files keep the training side's internal names in their metadata (size is small / medium / large, train_run is s2_a_small etc.); Re:Kord tells the sizes apart by file name.

Which size

The streaming budget is 0.256 s per block (8 frames × 32 ms). Per block (CQT + model, batch 1, median) on the author's PC:

	DirectML (discrete GPU)	CPU, 4 threads	CPU, 8 threads
S	7.9 ms	38.5 ms	30.3 ms
M	10.7 ms	118.9 ms	88.9 ms
L	17.2 ms	392 ms	278 ms

Model only, on the integrated GPU of the same PC (Core Ultra 9 285K, DirectML device 1): S about 25 ms, M about 80 ms, L about 273 ms. Times on weak CPUs and integrated GPUs are not measured.

Re:Kord's automatic choice follows from that: a discrete GPU runs L; an integrated GPU or a CPU with 8 or more threads runs M; a weaker CPU runs S. L exceeds the budget on the CPU and on an integrated GPU.

Accuracy

RWC-Pop 100 songs (full mixes as input, a cleaned reference, cross-validated thresholds, 0.26 s look-ahead). COnPOff (note onset and offset) per part and overall:

	S	M	L
vocal	.258	.305	.327
bass	.402	.444	.471
other	.166	.182	.198
all	.215	.235	.252

COnP (onset only, 0.15 s tolerance, all parts): S .584, M .604, L .618.

The vocal teacher of all three sizes is openvpi GAME's single notes. Compared with the earlier teacher, the vocal COnPOff went from .174 to .258 (S), .188 to .304 (M) and .193 to .331 (L); the vocal COnP at 50 ms from .382 to .531 (S), .406 to .575 (M) and .422 to .602 (L). Bass and other changed by at most 0.008 (M's bass COnPOff .382 to .374, S's .359 to .354, the rest within ±0.003).

The bass teacher of all three sizes is MuScriptor medium (run on a separated bass stem, keeping only notes it names as a bass instrument). Compared with the earlier bass teacher, the bass COnP at 50 ms went from .594 to .710 (S), .601 to .742 (M) and .602 to .758 (L); the bass COnPOff from .354 to .402 (S), .374 to .444 (M) and .385 to .471 (L); the bass pitch-class F-measure rose 3 to 5 points. Vocal and other changed by at most 0.005. Bass notes now start earlier and more of them are found.

Thresholds (onset / sustain)

Read from the ONNX metadata (thresholds_f8, thresholds_f16, thresholds_off; JSON, each part [th_on, th_sus]), not hard-coded. They were picked per part by 2-fold cross-validation on RWC-Pop, and then the sustain thresholds of bass and other were lowered by hand (to 0.50 and 0.20) because on RWC's MIDI notes, which are shorter than what actually sounds, cross-validation favors values that cut long held notes short.

	vocal	bass	other
S, 8 and 16 frames of look-ahead	.20 / .35	.25 / .50	.15 / .20
M, 8 and 16 frames	.25 / .45	.25 / .50	.15 / .20
L, 8 and 16 frames	.25 / .40	.25 / .50	.15 / .20

Known weakness: low solo instruments (contrabass, trombone, cello) come out worse than with a single-instrument model, and low notes can be split between the bass and other parts.

How Re:Kord streams them

• Every 8 frames (0.256 s) it runs one 5 s window that ends f frames after the block (f = 8 or 16), with audio after "now" zeroed and CQT frames before the start of the stream zeroed. Output rows [148 − f, 156 − f) of the time axis give the block's 8 frames (for every part).
• A note starts at an onset local maximum above the onset threshold and ends when sustain drops below the sustain threshold or at the next onset, per part with that part's thresholds.
• Always batch size 1: large on DirectML corrupts its output with bigger batches.

Training data

Dataset	License
MAESTRO v3	CC BY-NC-SA 4.0
GuitarSet	CC BY 4.0
GAPS (Hugging Face xavriley/GAPS)	MIT on Hugging Face; the Zenodo release is CC BY-NC-SA 4.0, non-commercial research use only
EGDB	no license stated
Slakh2100 (redux)	CC BY 4.0
URMP	no license stated (obtained through the dataset's request form)
GOAT (distorted electric guitar)	CC BY-NC 4.0
GAME 1.0 large (openvpi), used only as a labeler for the vocal part, not a training dataset	code: MIT (Copyright (c) Team OpenVPI); model files: CC BY-NC-SA 4.0
MuScriptor medium (Kyutai / Mirelo, arXiv:2607.08168), used only as a labeler for the bass part, not a training dataset	model weights: CC BY-NC 4.0

GAME (https://github.com/openvpi/GAME) transcribed the singing voice of the CD collection described below; its notes became the vocal labels of all three sizes. Its weights are not included here. Okarina is released under the same CC BY-NC-SA 4.0 and for non-commercial use only, so the two licenses are compatible.

MuScriptor (arXiv:2607.08168) transcribed the bass stem of the same CD collection (separated with htdemucs_ft); its notes named as a bass instrument became the bass labels of all three sizes. Its weights are not included here. Like GOAT, it is licensed CC BY-NC 4.0, for non-commercial use only, which Okarina's non-commercial license respects.

MAPS (CC BY-NC-SA 2.0) was used for evaluation only, not for training.

All three sizes were trained on these datasets (including GOAT), starting from a single-part model per size trained on them, and then on full mixes of the author's own collection of commercially released CDs with pseudo-labels. The pseudo-labels were made by a large model trained on those datasets including GOAT. (For the vocal part the pseudo-labels were later replaced by openvpi GAME's single-note transcription, and for the bass part by MuScriptor's; other is unchanged. See the GAME and MuScriptor rows above.) No audio from the CD collection is distributed; only the trained weights are. RWC-Pop was used for choosing thresholds and for evaluation, not for training.

Beat-tracking files (not Okarina)

beatthis_small0.onnx and logmel.onnx are Beat This! small0 (CPJKU, JKU Linz, https://github.com/CPJKU/beat_this) and its log-mel front end, converted from the published PyTorch checkpoint to ONNX by Re:Kord's export_beat_models.py, unchanged otherwise. They are here only so the Re:Kord app can download them on first launch, and are under the MIT license of Beat This! (beatthis_small0.LICENSE.txt), not under the license below.

License

Because the training data include material licensed for non-commercial use only, the Okarina weights (cqt156.onnx and okarina_*.onnx) are provided for non-commercial use only, under CC BY-NC-SA 4.0. They are not covered by Re:Kord's MIT license.