Instruction Following on IFEval (accuracy (greedy))
98.9Accuracy (Greedy)Gemma 4 31B
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemma 4 31BModel size=31B, Thinking mode=true2026.07 | 98.9 | |
| Gemma 4 26B-A4BModel size=26B-A4B, Thinking mode=true2026.07 | 98.5 | |
| Gemma 4 12BModel size=12B, Thinking mode=true2026.07 | 97.2 | |
| Gemma 4 E4BModel size=E4B, Thinking mode=true2026.07 | 96.7 | |
| Gemma 4 E2BModel size=E2B, Thinking mode=true2026.07 | 94.6 | |
| Gemma 3 27BModel size=27B, Thinking mode=false2026.07 | 90.4 | |
| Qwen3-30B-A3B-Instruct-2507# Bits=BF162026.05 | 83.18 | |
| FlexRound+LFQ# Bits=W4g1282026.05 | 82.99 | |
| FlexRound# Bits=W4g1282026.05 | 82.44 | |
| GPTQ# Bits=W4g1282026.05 | 82.26 | |
| gemma-3-27b-itType / λ=Base IT2025.10 | 80.59 | |
| gemma3-27b-slerpType / λ=0.92025.10 | 77.63 | |
| gemma-3-12b-itType / λ=Base IT2025.10 | 77.08 | |
| gemma3-12b-slerpType / λ=0.92025.10 | 76.16 | |
| gemma3-27b-slerpType / λ=0.82025.10 | 75.6 | |
| gemma3-12b-linearType / λ=0.92025.10 | 75.05 | |
| gemma3-12b-dare_tiesType / λ=0.92025.10 | 73.75 | |
| gemma3-12b-linearType / λ=0.82025.10 | 67.65 | |
| gemma3-12b-slerpType / λ=0.82025.10 | 67.1 | |
| gemma3-12b-dare_tiesType / λ=0.82025.10 | 65.99 | |
| gemma3-27b-slerpType / λ=0.72025.10 | 63.96 | |
| gemma3-12b-dare_tiesType / λ=0.72025.10 | 56.19 | |
| gemma3-12b-slerpType / λ=0.72025.10 | 55.45 | |
| gemma3-12b-linearType / λ=0.72025.10 | 55.45 | |
| gemma3-27b-slerpType / λ=0.62025.10 | 52.31 | |
| gemma3-27b-slerpType / λ=0.52025.10 | 51.94 | |
| gemma3-12b-dare_tiesType / λ=0.62025.10 | 50.09 | |
| gemma3-12b-linearType / λ=0.62025.10 | 49.17 | |
| gemma3-12b-dare_tiesType / λ=0.42025.10 | 48.98 | |
| gemma3-12b-slerpType / λ=0.62025.10 | 48.24 | |
| gemma3-27b-slerpType / λ=0.42025.10 | 48.24 | |
| gemma3-12b-slerpType / λ=0.52025.10 | 47.5 | |
| gemma3-12b-linearType / λ=0.52025.10 | 46.4 | |
| gemma3-12b-dare_tiesType / λ=0.52025.10 | 45.66 | |
| gemma3-12b-linearType / λ=0.42025.10 | 41.4 | |
| gemma3-12b-slerpType / λ=0.42025.10 | 40.67 | |
| gemma3-27b-slerpType / λ=0.32025.10 | 40.67 | |
| gemma3-12b-slerpType / λ=0.32025.10 | 35.49 | |
| gemma3-12b-dare_tiesType / λ=0.32025.10 | 35.3 | |
| gemma3-12b-linearType / λ=0.32025.10 | 34.75 | |
| gemma3-27b-slerpType / λ=0.22025.10 | 30.68 | |
| gemma3-27b-linearType / λ=0.22025.10 | 30.5 | |
| gemma3-12b-slerpType / λ=0.22025.10 | 26.62 | |
| gemma3-12b-linearType / λ=0.22025.10 | 26.43 | |
| gemma3-12b-dare_tiesType / λ=0.22025.10 | 25.88 | |
| gemma3-27b-linearType / λ=0.12025.10 | 24.58 | |
| gemma3-27b-slerpType / λ=0.12025.10 | 22.92 | |
| gemma3-12b-linearType / λ=0.12025.10 | 22 | |
| gemma3-12b-slerpType / λ=0.12025.10 | 20.89 | |
| gemma3-12b-dare_tiesType / λ=0.12025.10 | 20.52 | |
| gemma-3-27b-ptType / λ=Base PT2025.10 | 20.33 | |
| gemma-3-12b-ptType / λ=Base PT2025.10 | 19.41 |