Multi-task Language Understanding on MMLU-PRO (Acc %, ECE)
51.91Accuracy (%)gemma3-27b-slerp
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| gemma3-27b-slerpType / λ=0.52025.10 | 51.91 | 0.219 | |
| gemma3-27b-slerpType / λ=0.42025.10 | 51.46 | 0.192 | |
| gemma3-27b-slerpType / λ=0.32025.10 | 51.44 | 0.156 | |
| gemma3-27b-slerpType / λ=0.62025.10 | 51.11 | 0.254 | |
| gemma3-27b-slerpType / λ=0.72025.10 | 50.96 | 0.284 | |
| gemma3-27b-slerpType / λ=0.22025.10 | 50.89 | 0.119 | |
| gemma3-27b-linearType / λ=0.22025.10 | 50.84 | 0.122 | |
| gemma3-27b-slerpType / λ=0.82025.10 | 50.58 | 0.312 | |
| gemma3-27b-slerpType / λ=0.12025.10 | 50.41 | 0.077 | |
| gemma3-27b-linearType / λ=0.12025.10 | 50.23 | 0.078 | |
| gemma3-27b-slerpType / λ=0.92025.10 | 49.78 | 0.341 | |
| gemma-3-27b-ptType / λ=Base PT2025.10 | 49.39 | 0.037 | |
| gemma-3-27b-itType / λ=Base IT2025.10 | 47.8 | 0.478 | |
| gemma3-12b-slerpType / λ=0.42025.10 | 44.25 | 0.219 | |
| gemma3-12b-linearType / λ=0.42025.10 | 44.17 | 0.221 | |
| gemma3-12b-linearType / λ=0.32025.10 | 44.12 | 0.173 | |
| gemma3-12b-slerpType / λ=0.32025.10 | 44.11 | 0.173 | |
| gemma3-12b-slerpType / λ=0.52025.10 | 44.07 | 0.265 | |
| gemma3-12b-linearType / λ=0.52025.10 | 44.04 | 0.266 | |
| gemma3-12b-dare_tiesType / λ=0.22025.10 | 44 | 0.122 | |
| gemma3-12b-linearType / λ=0.22025.10 | 43.97 | 0.126 | |
| gemma3-12b-slerpType / λ=0.22025.10 | 43.94 | 0.125 | |
| gemma3-12b-linearType / λ=0.62025.10 | 43.82 | 0.312 | |
| gemma3-12b-dare_tiesType / λ=0.42025.10 | 43.79 | 0.232 | |
| gemma3-12b-slerpType / λ=0.62025.10 | 43.76 | 0.311 | |
| gemma3-12b-dare_tiesType / λ=0.32025.10 | 43.64 | 0.179 | |
| gemma3-12b-dare_tiesType / λ=0.12025.10 | 43.36 | 0.075 | |
| gemma3-12b-dare_tiesType / λ=0.62025.10 | 43.32 | 0.302 | |
| gemma3-12b-dare_tiesType / λ=0.52025.10 | 43.26 | 0.282 | |
| gemma3-12b-linearType / λ=0.72025.10 | 43.24 | 0.353 | |
| gemma3-12b-slerpType / λ=0.72025.10 | 43.22 | 0.352 | |
| gemma3-12b-slerpType / λ=0.12025.10 | 43.19 | 0.079 | |
| gemma3-12b-linearType / λ=0.12025.10 | 43.03 | 0.078 | |
| gemma3-12b-dare_tiesType / λ=0.72025.10 | 42.5 | 0.337 | |
| gemma-3-12b-ptType / λ=Base PT2025.10 | 42.35 | 0.024 | |
| gemma3-12b-linearType / λ=0.82025.10 | 42.15 | 0.395 | |
| gemma3-12b-slerpType / λ=0.82025.10 | 42.13 | 0.395 | |
| gemma3-12b-dare_tiesType / λ=0.82025.10 | 41.52 | 0.406 | |
| gemma3-12b-slerpType / λ=0.92025.10 | 41.43 | 0.43 | |
| gemma3-12b-linearType / λ=0.92025.10 | 41.36 | 0.431 | |
| gemma3-12b-dare_tiesType / λ=0.92025.10 | 40.43 | 0.448 | |
| gemma-3-12b-itType / λ=Base IT2025.10 | 39.82 | 0.533 |