Language Modeling on Open LLM Leaderboard Unified General Ability Suite (test)
12MMLU-P AccuracyHyperGLU
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| HyperGLUModel Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 12 | 26.8 | 32.1 | 1.2 | 46.8 | 28.6 | 18.9 | 36.6 | 61.1 | 51.6 | 70.8 | 61.3 | 53.1 | 76 | 35.3 | 78.8 | 1.88 | 9 | |
| HyperMLPModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.9 | 26.2 | 30 | 1 | 46.6 | 27.3 | 15.5 | 28.8 | 54.1 | 40.2 | 66.3 | 58.5 | 51.3 | 71 | 32 | 75.5 | 2.97 | 4 | |
| HyperGLUModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.7 | 25.9 | 31.3 | 0.9 | 45.1 | 27.3 | 15.7 | 30.4 | 53.8 | 42.7 | 67.4 | 58.7 | 51.8 | 72 | 33.6 | 77.1 | 2.31 | 6 | |
| HGRN2Model Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.5 | 25.4 | 29.5 | 0.2 | 35 | 22.3 | 12.9 | 28.2 | 50.4 | 31.7 | 67.1 | 41.6 | 52.2 | 77 | 32.8 | 37.8 | 5.84 | 1 | |
| HGRNModel Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.4 | 26.9 | 29.7 | 0.8 | 40.9 | 25.3 | 12.2 | 27.1 | 51.8 | 48.1 | 70.7 | 58.4 | 51.5 | 70 | 32.6 | 69.5 | 5.12 | 0 | |
| SoftmaxAttnModel Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.4 | 25.9 | 29.6 | 1.1 | 36.5 | 27 | 14.1 | 28 | 49.2 | 49.2 | 70.5 | 62.1 | 55.2 | 76 | 31.8 | 76.9 | 4.22 | 2 | |
| GLAModel Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.4 | 25.9 | 29.5 | 0.6 | 42.7 | 27.2 | 15.7 | 27.7 | 48.2 | 48.8 | 70.2 | 57.4 | 54.1 | 69 | 32.6 | 72.1 | 5.19 | 0 | |
| GatedDeltaNetModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.3 | 26 | 29.6 | 1 | 42.1 | 25.8 | 13.3 | 27.6 | 52.7 | 39.6 | 66.2 | 58.8 | 52.7 | 71 | 33.8 | 73.5 | 3.81 | 2 | |
| DeltaNetModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.2 | 26 | 30 | 0.9 | 45.2 | 27.7 | 15 | 26.9 | 50.2 | 40.5 | 65.3 | 51.9 | 50.4 | 69 | 31.6 | 71.7 | 4.81 | 1 | |
| ReLUAttn+KVConvModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11.1 | 25.3 | 29 | 0.8 | 42.3 | 22.9 | 13 | 26.8 | 52.5 | 41.2 | 66.9 | 52.4 | 52 | 66 | 32.4 | 77.4 | 5.13 | 1 | |
| GSAModel Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11 | 27 | 29.4 | 1.3 | 43.8 | 30 | 17.9 | 28.7 | 52.9 | 51 | 71.2 | 54.1 | 53.6 | 76 | 33 | 77.3 | 2.84 | 4 | |
| RetNetModel Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11 | 25.2 | 29.3 | 0.1 | 38.4 | 5.6 | 2.4 | 27.1 | 48.9 | 48 | 70.1 | 58.3 | 53.3 | 71 | 32.4 | 73.6 | 6.69 | 0 | |
| GLAModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 11 | 25.8 | 28.9 | 0.7 | 41.5 | 22.8 | 10.9 | 24.7 | 47.8 | 36.6 | 63.7 | 54.7 | 48.9 | 64 | 29.4 | 64.9 | 7.56 | 0 | |
| DeltaNetModel Parameters=1.3B, Training Tokens=100B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 10.9 | 26.3 | 30.8 | 1.1 | 41.7 | 28.8 | 16.5 | 26.6 | 52.2 | 50.2 | 70.4 | 61.1 | 54.1 | 74 | 31.8 | 76.1 | 4.22 | 0 | |
| DiffTransModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 10.9 | 25.9 | 29.9 | 0.8 | 39 | 26.6 | 13.3 | 28.9 | 53.1 | 40.8 | 66.8 | 60.3 | 53.4 | 69 | 33 | 73.4 | 4.13 | 2 | |
| SoftmaxAttnModel Parameters=340M, Training Tokens=15B, Shots=MMLU-P: 5, GPQA: 0, BBH: 3, MATH: 4, MuSR: 0; others 0-shot2026.02 | 10.6 | 26.7 | 29.2 | 1 | 38.6 | 26.9 | 12.6 | 27.3 | 50.6 | 39.6 | 65 | 56.9 | 49.9 | 72 | 32.4 | 72.7 | 5.28 | 3 |