Acoustic and paralinguistic modeling on SALMon
97.2Acoustic Consistency (Sentence)Human
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HumanParams=–, Textually Pretrained=–, Codebooks=–2026.03 | 97.2 | 91.5 | 98.6 | 93.3 | — | |
| LLaMA-Mimi 1.3BParams=1.3B, Textually Pretrained=✓, Codebooks=4 × 20482026.03 | 79 | 85 | 83.5 | 48.5 | 69 | |
| WT-SMA-24Params=∼357M, Textually Pretrained=✓, Codebooks=1 × 40962026.03 | 78.5 | 69 | 81.5 | 50.5 | 62.6 | |
| LLaMA-Mimi 8BParams=8.0B, Textually Pretrained=✓, Codebooks=4 × 20482026.03 | 76.5 | 86.5 | 85.5 | 46.5 | 69.5 | |
| WavSLM-2kParams=305M, Textually Pretrained=✗, Codebooks=1 × 20482026.03 | 75.5 | 86 | 89.5 | 45.5 | 68.3 | |
| WavSLM-4kParams=307M, Textually Pretrained=✗, Codebooks=1 × 40962026.03 | 75 | 88.5 | 90.5 | 51.5 | 69.5 | |
| WavSLM-65kParams=370M, Textually Pretrained=✗, Codebooks=1 × 655362026.03 | 75 | 87.5 | 84 | 45 | 66.5 | |
| SpiRit LM Expr.Params=7.0B, Textually Pretrained=✓, Codebooks=[501, 64, 100]2026.03 | 73.5 | 81 | 85 | 52 | 69.4 | |
| Mimi-S-24Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 20482026.03 | 71.5 | 78 | 77.5 | 52 | 64 | |
| DWavL-S-16Params=∼357M, Textually Pretrained=✓, Codebooks=6 × 10002026.03 | 70 | 86.5 | 92 | 49 | 69 | |
| LAST 1.3BParams=1.3B, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 65 | 64.5 | 68.5 | 53.5 | — | |
| SQ-SMA-16Params=∼357M, Textually Pretrained=✓, Codebooks=4 × 196832026.03 | 64 | 84.5 | 83 | 50.5 | 62.9 | |
| HuBERT 25HzParams=∼357M, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 62.5 | 69 | 69.5 | 53 | 65.3 | |
| TWIST 1.3BParams=1.3B, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 61.5 | 69 | 69.5 | 53 | 64.8 | |
| TWIST 7BParams=7.0B, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 61.5 | 71 | 70 | 51.5 | 66.3 | |
| DAC-SMA-16Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 10242026.03 | 60 | 77 | 81 | 50 | 60.2 | |
| ST-S-16Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 10242026.03 | 58 | 65 | 66.5 | 49.5 | 57.2 | |
| Enc-SMA-24Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 10242026.03 | 56.5 | 65 | 70.5 | 50 | 56.1 | |
| SpiRit LMParams=7.0B, Textually Pretrained=✓, Codebooks=1 × 5012026.03 | 54.5 | 69.5 | 67 | 48 | 64.2 |