General Language Understanding on 12-task evaluation suite (test)
71.62Average ScoreEfficient-DLM 8B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Efficient-DLM 8BType=dLM, TPF=12025.12 | 71.62 | — | — | — | — | 39.99 | 67.36 | 69.22 | 77.22 | 74.88 | |
| Qwen3 8BType=AR, TPF=12025.12 | 71.58 | — | — | — | — | 42.51 | 68.45 | 69.87 | 76.93 | 73.71 | |
| Efficient-DLM 8BType=dLM, TPF=2.572025.12 | 70.93 | — | — | — | — | 103.89 | 65.64 | 68.52 | 77.22 | 74.88 | |
| Efficient-DLM 8BType=dLM, TPF=3.12025.12 | 70.65 | — | — | — | — | 126.43 | 64.95 | 68.21 | 77.22 | 74.88 | |
| Qwen3 4BType=AR, TPF=12025.12 | 67.97 | — | — | — | — | 47.13 | 63.85 | 66.27 | 73.19 | 70.91 | |
| Efficient-DLM 4BType=dLM, TPF=12025.12 | 67.54 | — | — | — | — | 44.13 | 62.08 | 67.98 | 71.8 | 70.87 | |
| Efficient-DLM 4BType=dLM, TPF=2.522025.12 | 67.39 | — | — | — | — | 119.33 | 61.37 | 68.56 | 71.8 | 70.87 | |
| Efficient-DLM 4BType=dLM, TPF=3.012025.12 | 67.18 | — | — | — | — | 130.24 | 60.96 | 68.1 | 71.8 | 70.87 | |
| Dream 7BType=dLM, TPF=12025.12 | 65.3 | — | — | — | — | 28.11 | 58.92 | 58.39 | 67 | 72.83 | |
| FineWeb-MaskLarge Scale Corpus=FineWeb-Mask, Model Architecture=7B MoE2025.12 | 60.2 | 57.5 | 66.4 | 54.4 | 912 | — | — | — | — | — | |
| Qwen3 1.7BType=AR, TPF=12025.12 | 59.39 | — | — | — | — | 71.59 | 54.22 | 54.15 | 62.53 | 64.99 | |
| FineWeb-EduLarge Scale Corpus=FineWeb-Edu, Model Architecture=7B MoE2025.12 | 59.1 | 57.4 | 66.2 | 53 | 312 | — | — | — | — | — | |
| LLaDA 8BType=dLM, TPF=12025.12 | 54.92 | — | — | — | — | 25.04 | 38.1 | 49.13 | 65.86 | 68.5 | |
| Qwen2.5 1.5BType=AR, TPF=12025.12 | 54.47 | — | — | — | — | 73.03 | 42.17 | 46.98 | 60.96 | 66 | |
| Efficient-DLM 1.5BType=dLM, TPF=12025.12 | 52.09 | — | — | — | — | 68.52 | 42.33 | 42.6 | 57.63 | 62.58 | |
| Efficient-DLM 1.5BType=dLM, TPF=2.332025.12 | 52.04 | — | — | — | — | 158.89 | 42.33 | 42.28 | 57.63 | 62.58 | |
| Efficient-DLM 1.5BType=dLM, TPF=2.692025.12 | 51.77 | — | — | — | — | 184.48 | 41.79 | 41.8 | 57.63 | 62.58 | |
| Qwen2.5 0.5BType=AR, TPF=12025.12 | 41.98 | — | — | — | — | 99.93 | 31.9 | 25.97 | 47.65 | 55.31 | |
| SmolLM2 1.7BType=AR, TPF=12025.12 | 40.14 | — | — | — | — | 112.84 | 21.06 | 30.97 | 49.99 | 68.44 | |
| Llama3.2 1BType=AR, TPF=12025.12 | 34.24 | — | — | — | — | 143.91 | 24.45 | 4.98 | 30.98 | 60.62 |