List your company
ResearchFormerly WizWand

Multi-task Language Understanding on MMLU-PRO (Acc %, ECE)

51.91Accuracy (%)gemma3-27b-slerp
39.336442.600745.86549.1293Oct 20, 2025
Updated 4mo ago

Evaluation results

MethodLinks
gemma3-27b-slerpType / λ=0.52025.1051.910.219
gemma3-27b-slerpType / λ=0.42025.1051.460.192
gemma3-27b-slerpType / λ=0.32025.1051.440.156
gemma3-27b-slerpType / λ=0.62025.1051.110.254
gemma3-27b-slerpType / λ=0.72025.1050.960.284
gemma3-27b-slerpType / λ=0.22025.1050.890.119
gemma3-27b-linearType / λ=0.22025.1050.840.122
gemma3-27b-slerpType / λ=0.82025.1050.580.312
gemma3-27b-slerpType / λ=0.12025.1050.410.077
gemma3-27b-linearType / λ=0.12025.1050.230.078
gemma3-27b-slerpType / λ=0.92025.1049.780.341
gemma-3-27b-ptType / λ=Base PT2025.1049.390.037
gemma-3-27b-itType / λ=Base IT2025.1047.80.478
gemma3-12b-slerpType / λ=0.42025.1044.250.219
gemma3-12b-linearType / λ=0.42025.1044.170.221
gemma3-12b-linearType / λ=0.32025.1044.120.173
gemma3-12b-slerpType / λ=0.32025.1044.110.173
gemma3-12b-slerpType / λ=0.52025.1044.070.265
gemma3-12b-linearType / λ=0.52025.1044.040.266
gemma3-12b-dare_tiesType / λ=0.22025.10440.122
gemma3-12b-linearType / λ=0.22025.1043.970.126
gemma3-12b-slerpType / λ=0.22025.1043.940.125
gemma3-12b-linearType / λ=0.62025.1043.820.312
gemma3-12b-dare_tiesType / λ=0.42025.1043.790.232
gemma3-12b-slerpType / λ=0.62025.1043.760.311
gemma3-12b-dare_tiesType / λ=0.32025.1043.640.179
gemma3-12b-dare_tiesType / λ=0.12025.1043.360.075
gemma3-12b-dare_tiesType / λ=0.62025.1043.320.302
gemma3-12b-dare_tiesType / λ=0.52025.1043.260.282
gemma3-12b-linearType / λ=0.72025.1043.240.353
gemma3-12b-slerpType / λ=0.72025.1043.220.352
gemma3-12b-slerpType / λ=0.12025.1043.190.079
gemma3-12b-linearType / λ=0.12025.1043.030.078
gemma3-12b-dare_tiesType / λ=0.72025.1042.50.337
gemma-3-12b-ptType / λ=Base PT2025.1042.350.024
gemma3-12b-linearType / λ=0.82025.1042.150.395
gemma3-12b-slerpType / λ=0.82025.1042.130.395
gemma3-12b-dare_tiesType / λ=0.82025.1041.520.406
gemma3-12b-slerpType / λ=0.92025.1041.430.43
gemma3-12b-linearType / λ=0.92025.1041.360.431
gemma3-12b-dare_tiesType / λ=0.92025.1040.430.448
gemma-3-12b-itType / λ=Base IT2025.1039.820.533