Discourse-level coherence evaluation on Topic Story-Cloze (tSC)
87.6tSC ScoreSmolTolk 2B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SmolTolk 2BParams=2.0B, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 87.6 | — | |
| Moshi 7BParams=7.7B, Textually Pretrained=✓, Codebooks=8 × 20482026.03 | 83 | — | |
| SpiRit LMParams=7.0B, Textually Pretrained=✓, Codebooks=1 × 5012026.03 | 82.9 | 64.2 | |
| TWIST 7BParams=7.0B, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 76.4 | 66.3 | |
| SpiRit LM Expr.Params=7.0B, Textually Pretrained=✓, Codebooks=[501, 64, 100]2026.03 | 75.4 | 69.4 | |
| HuBERT 25HzParams=∼357M, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 71.5 | 65.3 | |
| TWIST 1.3BParams=1.3B, Textually Pretrained=✓, Codebooks=1 × 5002026.03 | 70.6 | 64.8 | |
| LLaMA-Mimi 8BParams=8.0B, Textually Pretrained=✓, Codebooks=4 × 20482026.03 | 67.6 | 69.5 | |
| LLaMA-Mimi 1.3BParams=1.3B, Textually Pretrained=✓, Codebooks=4 × 20482026.03 | 64 | 69 | |
| WavSLM-4kParams=307M, Textually Pretrained=✗, Codebooks=1 × 40962026.03 | 63.3 | 69.5 | |
| WavSLM-2kParams=305M, Textually Pretrained=✗, Codebooks=1 × 20482026.03 | 62.9 | 68.3 | |
| DWavL-S-16Params=∼357M, Textually Pretrained=✓, Codebooks=6 × 10002026.03 | 62.4 | 69 | |
| WavSLM-65kParams=370M, Textually Pretrained=✗, Codebooks=1 × 655362026.03 | 60.4 | 66.5 | |
| ST-S-16Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 10242026.03 | 55.7 | 57.2 | |
| SQ-SMA-16Params=∼357M, Textually Pretrained=✓, Codebooks=4 × 196832026.03 | 55.1 | 62.9 | |
| Mimi-S-24Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 20482026.03 | 54.3 | 64 | |
| WT-SMA-24Params=∼357M, Textually Pretrained=✓, Codebooks=1 × 40962026.03 | 52.8 | 62.6 | |
| DAC-SMA-16Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 10242026.03 | 51.5 | 60.2 | |
| Enc-SMA-24Params=∼357M, Textually Pretrained=✓, Codebooks=8 × 10242026.03 | 48.2 | 56.1 |