Common-sense Reasoning on WikiText, LAMBADA, PIQA, HellaSwag, WinoGrande, and ARC
25.89PPL (WikiText)NaLa+GDN
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| NaLa+GDNParameter size=340M2025.06 | 25.89 | 32.32 | 65.6 | 36.2 | 53.2 | 45.4 | 23.8 | 44.8 | |
| Gated DeltaNetParameter size=340M2025.06 | 26.59 | 31.67 | 65.8 | 35.2 | 50.8 | 46 | 23.5 | 44.3 | |
| NaLa+DNParameter size=340M2025.06 | 27.82 | 49.77 | 64.9 | 34.3 | 52.7 | 46.5 | 23.1 | 44.3 | |
| Transformer++Parameter size=340M2025.06 | 28.39 | 42.69 | 63.3 | 34 | 50.4 | 44.5 | 24.2 | 43.3 | |
| MambaParameter size=340M2025.06 | 28.39 | 39.66 | 65 | 35.4 | 50.1 | 46.3 | 23.6 | 44.1 | |
| GLAParameter size=340M2025.06 | 28.65 | 43.35 | 64.8 | 34.5 | 51.4 | 45.1 | 22.7 | 43.7 | |
| DeltaNetParameter size=340M2025.06 | 29.08 | 50.87 | 63.6 | 33.6 | 51.7 | 46 | 23 | 43.6 | |
| RetNetParameter size=340M2025.06 | 32.33 | 49.19 | 63.5 | 33.5 | 52.5 | 44.5 | 23.4 | 43.5 |