Model card — Music3Lab adapters & encoders
This card covers the trained checkpoints produced by Music3Lab. The base
MiniMax-Music3 model is not covered here — see its own card at
MiniMaxAI/MiniMax-Music3.
⚠️ License: these checkpoints are likely derivative works of MiniMax-Music3 (trained through its frozen decoder and/or on its captured state). Their redistribution may be governed by the MiniMax-Music3 license. Confirm before publishing. See THIRD_PARTY.md. The checkpoints are not in this Git repo; only pointers are (
checkpoints/README.md).
Common facts
- Base model: MiniMax-Music3 (frozen; never fine-tuned by this project).
- What stays frozen: the DAV/Flow decoder, the Global/Local language models, and the vocoder. Only the small adapters/encoders below are trained.
- Training compute: single NVIDIA H100 80 GB.
- Evaluation: preregistered objective gates only (SI-SDR, SNR, correlation, reconstruction ruler, latent NMSE, loudness/stereo, boundary continuity, anti-copy). No human listening tests. No learned aesthetic reward.
- Author's own songs were held out of all training and checkpoint selection (out-of-distribution evaluation only).
Checkpoints
| Checkpoint | Arch / trainable params | Trained on | Gate result |
|---|---|---|---|
flow-encoder (base) |
WAV [B,2,44032] → latent [B,128,86], Conv1d frontend (~1M) |
Music3-generated WAV/latent teacher pairs | ✅ pilot pass; ~1.34 ms one-pass |
external-finetune encoder |
same arch, fine-tuned | 542 LAION real-music + Music3 teachers | 🟡 rejected specialist — external held-out ruler +74.9%, SI-SDR 2.03→8.24 dB, but protected teacher latent NMSE regressed +13.2% (> 5% limit) |
masked-flow-inpaint adapter |
rank-4 LoRA on 108 QKV projs × 36 Flow blocks + 3 embeddings = 1,775,616 | captured Music3 conditions | ✅ pilot: +30.8% latent NMSE, +20.1% hole ruler (captured-condition only) |
learned-continuation / -residual adapter |
rank-4 Flow LoRA (~1.7M) | adjacent real-audio windows | ❌ lost to repeat/roll baselines |
native-token adapter |
1×16384 semantic head + 7×1024 residual heads | 96 Music3 token captures | ❌ residual CE regressed; not a tokenizer |
acoustic-fim-v2 |
2-sided waveform U-Net, 4,063,090 | 542 LAION real-music | ❌ +5.9% vs +10% gate |
audio-prepend / waveform-right-context-prepend |
spectral/attention (~5.4M) | real-audio right-context | ❌ seam / anti-copy gates failed |
waveform-causal-continuation |
causal waveform net | real-audio history | ❌ collapsed to near-exact repeat-tail copy |
native-state-stage0 (v3) |
3-branch posterior (log-Mel + stereo STFT + latent), 8 Conformer blocks | one Music3 clip | ✅ one-clip 25/25 alignment (feedback NMSE 5.1e-13) — interpolation, not a general encoder |
native-state-distill |
c0 soft-logit distillation | 104 captured clips | 🟡 bounded pass (held-out hard CE 8.97); c1–c7 absent |
native-stage1-residual |
autoregressive c1–c7 heads | 1,024 captured clips | ❌ near-modal; 0 exact rows |
official_local_audio_heads |
frozen official Local heads, extracted | (extracted from base) | support artifact so native-state code runs without the 57 GB model |
Exact per-run commit hashes, selected steps, validation losses, and artifact
SHA-256s are in reports/RELEASE_STATUS.md and
reports/FINAL_RESULTS.md.
Intended use
Research and reproduction: studying continuous-latent audio representations, editing in Music3's Flow-latent space, capture/replay of generation state, and objective evaluation methodology — including studying the negative results.
Out-of-scope / limitations
- Not a native WAV→token encoder for Music3 (that is unsolved with the released weights).
- Continuous inversion is a slow research teacher (~208 s per 1 s), not real-time.
- Arbitrary-WAV continuation / inpainting / prepend do not work at the target quality; those checkpoints are provided as reproducible negative results.
- No safety/aesthetic/musicality guarantees. Objective metrics ≠ musical quality.