Speech Generation on LibriSpeech (test-clean)
0.966Speaker SimilarityVAE reconstruction (oracle latent)
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| VAE reconstruction (oracle latent)Dim=242026.06 | 0.966 | 4.1 | — | — | — | — | 3.67 | — | — | 2.14 | |
| CosyVoice2 Token2Wavsampling_steps=10-step2026.06 | 0.94 | 4.05 | — | — | — | — | 3.76 | — | 0.0775 | 3.18 | |
| Latent MeanFlow + VAERefinement=Joint-FT, Dim=242026.06 | 0.932 | 3.85 | — | — | — | — | 3.64 | — | 0.0046 | 3.41 | |
| Latent MeanFlow + VAERefinement=Joint-FT, Dim=162026.06 | 0.927 | 3.72 | — | — | — | — | 3.56 | — | 0.0046 | 3.62 | |
| WavSLM-2kParams=305M, Textually Pretrained=✗, Codebooks=1 × 20482026.03 | 0.918 | — | — | — | — | — | 3.72 | 161 | 5.9 | — | |
| WavSLM-4kParams=307M, Textually Pretrained=✗, Codebooks=1 × 40962026.03 | 0.916 | — | — | — | — | — | 3.69 | 162 | 5.8 | — | |
| LLaMA-Mimi 8BParams=8.0B, Textually Pretrained=✓, Codebooks=4 × 20482026.03 | 0.915 | — | — | — | — | — | 3.56 | 122 | 1.1 | — | |
| LLaMA-Mimi 1.3BParams=1.3B, Textually Pretrained=✓, Codebooks=4 × 20482026.03 | 0.913 | — | — | — | — | — | 3.57 | 153 | 2 | — | |
| WavSLM-65kParams=370M, Textually Pretrained=✗, Codebooks=1 × 655362026.03 | 0.913 | — | — | — | — | — | 3.66 | 210 | 5.8 | — | |
| Llama-Mimi-1.3BTrain Data Size (Hours)=240K, Tokenizer=RVQ2025.09 | 0.346 | — | — | — | — | 3.01 | — | — | — | — | |
| Flow-SLM-1B-extTrain Data Size (Hours)=110K, Tokenizer=RVQ2025.09 | 0.331 | — | — | — | — | 3.12 | — | — | — | — | |
| CSM-1.3BTrain Data Size (Hours)=240K, Tokenizer=RVQ2025.09 | 0.32 | — | — | — | — | 2.8 | — | — | — | — | |
| TWIST-1.3BTrain Data Size (Hours)=150K, Tokenizer=SSL2025.09 | 0.11 | — | — | — | — | 3.63 | — | — | — | — | |
| SpiritLM-exp-7BTrain Data Size (Hours)=570K, Tokenizer=SSL2025.09 | 0.078 | — | — | — | — | 3.69 | — | — | — | — | |
| SpiritLM-base-7BTrain Data Size (Hours)=570K, Tokenizer=SSL2025.09 | 0.035 | — | — | — | — | 4.43 | — | — | — | — | |
| MS-TLMID=9, Input=u, Output=(u, d, lf), Quantized=false, Units=HUBERT, Training Data=LL6k, Delta=12021.09 | — | 3.16 | 3.8 | 3.69 | — | — | — | — | — | — | |
| MS-TLMID=10, Input=u, Output=(u, d, lf), Quantized=true, Units=HUBERT, Training Data=LL6k, Delta=12021.09 | — | 2.66 | 3.36 | 3.15 | — | — | — | — | — | — | |
| MS-TLMID=11, Input=(u,d,lf), Output=(u, d, lf), Quantized=false, Units=HUBERT, Training Data=LL6k, Delta=12021.09 | — | 3.31 | 3.76 | 3.78 | — | — | — | — | — | — | |
| MS-TLMID=12, Input=(u,d,lf), Output=(u, d, lf), Quantized=true, Units=HUBERT, Training Data=LL6k, Delta=12021.09 | — | 3.43 | 4.04 | 3.75 | — | — | — | — | — | — | |
| resynthesized2021.09 | — | 3.21 | 3.95 | 3.87 | — | — | — | — | — | — |