# Model card — Music3Lab adapters & encoders

This card covers the trained checkpoints produced by Music3Lab. **The base
MiniMax-Music3 model is not covered here** — see its own card at
`MiniMaxAI/MiniMax-Music3`.

> ⚠️ **License:** these checkpoints are likely **derivative works of
> MiniMax-Music3** (trained through its frozen decoder and/or on its captured
> state). Their redistribution may be governed by the MiniMax-Music3 license.
> Confirm before publishing. See THIRD_PARTY.md. The checkpoints are **not** in
> this Git repo; only pointers are (`checkpoints/README.md`).

## Common facts

- **Base model:** MiniMax-Music3 (frozen; never fine-tuned by this project).
- **What stays frozen:** the DAV/Flow decoder, the Global/Local language models,
  and the vocoder. Only the small adapters/encoders below are trained.
- **Training compute:** single NVIDIA H100 80 GB.
- **Evaluation:** preregistered objective gates only (SI-SDR, SNR, correlation,
  reconstruction ruler, latent NMSE, loudness/stereo, boundary continuity,
  anti-copy). **No human listening tests. No learned aesthetic reward.**
- **Author's own songs** were held out of all training and checkpoint selection
  (out-of-distribution evaluation only).

## Checkpoints

| Checkpoint | Arch / trainable params | Trained on | Gate result |
|---|---|---|---|
| `flow-encoder` (base) | WAV `[B,2,44032]` → latent `[B,128,86]`, Conv1d frontend (~1M) | Music3-generated WAV/latent teacher pairs | ✅ pilot pass; ~1.34 ms one-pass |
| `external-finetune` encoder | same arch, fine-tuned | 542 LAION real-music + Music3 teachers | 🟡 **rejected specialist** — external held-out ruler +74.9%, SI-SDR 2.03→8.24 dB, but protected teacher latent NMSE regressed +13.2% (> 5% limit) |
| `masked-flow-inpaint` adapter | rank-4 LoRA on 108 QKV projs × 36 Flow blocks + 3 embeddings = 1,775,616 | captured Music3 conditions | ✅ pilot: +30.8% latent NMSE, +20.1% hole ruler (captured-condition only) |
| `learned-continuation` / `-residual` adapter | rank-4 Flow LoRA (~1.7M) | adjacent real-audio windows | ❌ lost to repeat/roll baselines |
| `native-token` adapter | 1×16384 semantic head + 7×1024 residual heads | 96 Music3 token captures | ❌ residual CE regressed; not a tokenizer |
| `acoustic-fim-v2` | 2-sided waveform U-Net, 4,063,090 | 542 LAION real-music | ❌ +5.9% vs +10% gate |
| `audio-prepend` / `waveform-right-context-prepend` | spectral/attention (~5.4M) | real-audio right-context | ❌ seam / anti-copy gates failed |
| `waveform-causal-continuation` | causal waveform net | real-audio history | ❌ collapsed to near-exact repeat-tail copy |
| `native-state-stage0` (v3) | 3-branch posterior (log-Mel + stereo STFT + latent), 8 Conformer blocks | one Music3 clip | ✅ one-clip 25/25 alignment (feedback NMSE 5.1e-13) — **interpolation, not a general encoder** |
| `native-state-distill` | c0 soft-logit distillation | 104 captured clips | 🟡 bounded pass (held-out hard CE 8.97); c1–c7 absent |
| `native-stage1-residual` | autoregressive c1–c7 heads | 1,024 captured clips | ❌ near-modal; 0 exact rows |
| `official_local_audio_heads` | frozen official Local heads, extracted | (extracted from base) | support artifact so native-state code runs without the 57 GB model |

Exact per-run commit hashes, selected steps, validation losses, and artifact
SHA-256s are in [`reports/RELEASE_STATUS.md`](reports/RELEASE_STATUS.md) and
[`reports/FINAL_RESULTS.md`](reports/FINAL_RESULTS.md).

## Intended use

Research and reproduction: studying continuous-latent audio representations,
editing in Music3's Flow-latent space, capture/replay of generation state, and
objective evaluation methodology — including studying the **negative** results.

## Out-of-scope / limitations

- Not a native WAV→token encoder for Music3 (that is unsolved with the released
  weights).
- Continuous inversion is a slow research teacher (~208 s per 1 s), not real-time.
- Arbitrary-WAV continuation / inpainting / prepend **do not work** at the target
  quality; those checkpoints are provided as reproducible negative results.
- No safety/aesthetic/musicality guarantees. Objective metrics ≠ musical quality.