Zero-shot Reasoning Suite (LAMBADA, ARC, HellaSwag, PIQA, WinoGrande)
57.3LAMBADA PPL ScoreOVQ/SWA
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| OVQ/SWAParameters=120M2026.05 | 57.3 | 29.9 | 26.1 | 49.5 | 32.4 | 63.4 | 50.1 | 41.9 | |
| GPTA-7 NoPE/SWAParameters=120M2026.05 | 56.5 | 29.7 | 24 | 49.5 | 32.8 | 64.1 | 51.9 | 42 | |
| KVM 256Parameters=120M2026.05 | 53.4 | 30.3 | 23.7 | 49.8 | 33.1 | 64.1 | 51 | 42 | |
| KVM/SWAParameters=120M2026.05 | 51.8 | 30.6 | 24.7 | 49.6 | 33.1 | 63.5 | 52.1 | 42.3 | |
| GPTA-2 HalfRoPE/SWAParameters=120M2026.05 | 51.6 | 31 | 23.5 | 50.2 | 33.5 | 64.7 | 50 | 42.1 | |
| GPTA-2Parameters=120M2026.05 | 51.5 | 31 | 24.7 | 49.2 | 32.8 | 64.4 | 50 | 42 | |
| KVM sqrtParameters=120M2026.05 | 51 | 31 | 25.1 | 51.1 | 33.1 | 64.1 | 51.6 | 42.7 | |
| BSWAParameters=120M2026.05 | 47.7 | 31.5 | 24.5 | 50.1 | 32.6 | 63.5 | 50.4 | 42.1 | |
| RWKV-7Parameters=120M2026.05 | 42.6 | 31.7 | 24.7 | 49.8 | 33.6 | 64.3 | 52.2 | 42.7 | |
| OVQ/SWAParameters=350M2026.05 | 23.2 | 38.8 | 25.4 | 56.9 | 41.5 | 67.7 | 51.4 | 46.9 | |
| GPTA-7Parameters=350M2026.05 | 22.9 | 38.2 | 27.1 | 56.2 | 41.7 | 68.2 | 51.5 | 47.2 | |
| GPTA-2 NoPE/SWAParameters=350M2026.05 | 22.9 | 38.6 | 27.6 | 57.1 | 41.7 | 68 | 51.1 | 47.4 | |
| BSWAParameters=350M2026.05 | 22.2 | 38.8 | 27 | 56.4 | 41.9 | 68.9 | 51.9 | 47.5 | |
| KVM 256Parameters=350M2026.05 | 22 | 38.2 | 27 | 56.9 | 41.9 | 68.6 | 51.3 | 47.3 | |
| KVM/SWAParameters=350M2026.05 | 22 | 38.8 | 26.9 | 57.7 | 42.7 | 68.6 | 49.4 | 47.3 | |
| KVM sqrtParameters=350M2026.05 | 21.7 | 38.8 | 27 | 57 | 42 | 69.1 | 52.2 | 47.7 | |
| GPTA-2 HalfRoPE/SWAParameters=350M2026.05 | 21.6 | 38.9 | 27.2 | 57 | 42.3 | 69 | 50.7 | 47.5 | |
| RWKV-7Parameters=350M2026.05 | 19.9 | 40.2 | 26.9 | 57.1 | 42.9 | 69.3 | 53 | 48.2 |