Last-word prediction on LAMBADA
17.3AccuracySISA
Evaluation Results
| Method | Links | |
|---|---|---|
| SISAmodel size=152M, decoding strategy=greedy, ds=16, mb=42026.06 | 17.3 | |
| SISAmodel size=152M, decoding strategy=greedy, ds=64, mb=42026.06 | 16.8 | |
| SISAmodel size=152M, decoding strategy=greedy, ds=128, mb=42026.06 | 16.8 | |
| SISAmodel size=152M, decoding strategy=greedy, ds=32, mb=22026.06 | 16.1 | |
| Mamba-3model size=152M, decoding strategy=greedy, mb=42026.06 | 15.5 | |
| Transformermodel size=152M, decoding strategy=greedy, mb=42026.06 | 13.9 | |
| Mamba-2model size=152M, decoding strategy=greedy, mb=42026.06 | 12.7 |