Multi-Query Associative Recall (MQAR) on MQAR Shuffle v1
99.99AccuracyPythia
Evaluation Results
| Method | Links | |
|---|---|---|
| PythiaScale=133m2024.10 | 99.99 | |
| PythiaScale=133m2024.10 | 99.82 | |
| Mamba w/ 4xState SizeScale=134m, State Size=4x2024.10 | 96.92 | |
| PythiaScale=133m2024.10 | 94.31 | |
| Mamba w/ Global SelectionScale=133m, Global Selection=true2024.10 | 90.45 | |
| Mamba w/ 2xState SizeScale=130m, State Size=2x2024.10 | 88.57 | |
| MambaScale=129m2024.10 | 80.98 | |
| Mamba w/ Global SelectionScale=133m, Global Selection=true2024.10 | 80.54 | |
| Mamba w/ Global SelectionScale=133m, Global Selection=true2024.10 | 35.73 | |
| Mamba w/ 4xState SizeScale=134m, State Size=4x2024.10 | 32.88 | |
| Mamba w/ 2xState SizeScale=130m, State Size=2x2024.10 | 31.88 | |
| HyenaScale=153m2024.10 | 22.51 | |
| MambaScale=129m2024.10 | 22.37 | |
| RWKVScale=153m2024.10 | 6.57 |