Recall-Intensive Evaluation on Recall-Intensive Tasks
69.48FDA AccuracyGLA-H
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GLA-HParameters=1.3B, Training Tokens=100B, Architecture Type=Hybrid2025.10 | 69.48 | 46.77 | 57.05 | 27.15 | 40.28 | 21.47 | 43.7 | |
| GSA-HParameters=1.3B, Training Tokens=100B, Architecture Type=Hybrid2025.10 | 68.3 | 51.83 | 58.59 | 26.42 | 42.36 | 22.42 | 44.99 | |
| NHAParameters=1.3B, Training Tokens=100B, Architecture Type=Hybrid2025.10 | 68.3 | 52.48 | 59.6 | 27.18 | 45.58 | 25.44 | 46.43 | |
| GDN-HParameters=1.3B, Training Tokens=100B, Architecture Type=Hybrid2025.10 | 66.76 | 49.02 | 59.06 | 29.3 | 41.15 | 24.01 | 44.88 | |
| Mamba2-HParameters=1.3B, Training Tokens=100B, Architecture Type=Hybrid2025.10 | 65.3 | 50.89 | 55.92 | 28.54 | 41.99 | 22.46 | 44.18 | |
| GSA-HParameters=340M, Training Tokens=15B, Architecture Type=Hybrid2025.10 | 62.13 | 45.36 | 43.78 | 20.62 | 31.17 | 18.78 | 36.97 | |
| GLA-HParameters=340M, Training Tokens=15B, Architecture Type=Hybrid2025.10 | 58.86 | 39.36 | 44.14 | 17.01 | 31.37 | 18.21 | 34.83 | |
| Mamba2-HParameters=340M, Training Tokens=15B, Architecture Type=Hybrid2025.10 | 55.68 | 39.55 | 45.5 | 18.34 | 27.98 | 17.63 | 34.11 | |
| NHAParameters=340M, Training Tokens=15B, Architecture Type=Hybrid2025.10 | 53.86 | 48.55 | 46.56 | 20.56 | 38.6 | 23.48 | 38.6 | |
| GDN-HParameters=340M, Training Tokens=15B, Architecture Type=Hybrid2025.10 | 52.32 | 38.8 | 39.4 | 15.24 | 29.83 | 17.92 | 32.25 | |
| Trans++Parameters=340M, Training Tokens=15B, Architecture Type=Base2025.10 | 46.14 | 25.87 | 45.97 | 18.94 | 33.22 | 20.03 | 31.7 | |
| Trans++Parameters=1.3B, Training Tokens=100B, Architecture Type=Base2025.10 | 44.32 | 32.43 | 58.47 | 24.49 | 42.59 | 21.56 | 37.31 | |
| GDNParameters=1.3B, Training Tokens=100B, Architecture Type=Base2025.10 | 30.25 | 27.65 | 58.23 | 23.22 | 34.06 | 20.36 | 32.3 | |
| GLAParameters=1.3B, Training Tokens=100B, Architecture Type=Base2025.10 | 27.61 | 30.93 | 56.28 | 22.27 | 35.04 | 19.45 | 31.93 | |
| GSAParameters=1.3B, Training Tokens=100B, Architecture Type=Base2025.10 | 23.25 | 32.8 | 57.05 | 22.96 | 35.57 | 20.65 | 32.05 | |
| GDNParameters=340M, Training Tokens=15B, Architecture Type=Base2025.10 | 20.53 | 23.24 | 44.91 | 14.98 | 28.55 | 16.48 | 24.78 | |
| GLAParameters=340M, Training Tokens=15B, Architecture Type=Base2025.10 | 11.26 | 16.78 | 43.9 | 12.77 | 27.85 | 17.68 | 21.71 | |
| GSAParameters=340M, Training Tokens=15B, Architecture Type=Base2025.10 | 6.36 | 16.87 | 42.18 | 14.6 | 21.9 | 16.72 | 19.77 |