MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU
1.33Loss (FT)
10
MMLU-Pro
47.72Accuracy
10
MMLU Clinical Knowledge
92Accuracy
10
MMLU (test)
35.91Dolly MMLU Score
10
MMLU (All)
28.7Accuracy
10
MMLU STEM (test)
75Accuracy (STEM Test)
10
MMLU-Pro Health discipline
9.38Mean EMR
10
MMLU-Pro
73.76Avg@8
9
MMLU-Pro (test)
16.2ECE
9
MMLU
76.63MMLU Score
9
Global-MMLU
70.4Accuracy
9
MMLU Topic Distributions
80Generalized Score
9
MMLU
78.12MMLU Utility
9
MMLU (test)
0.07ECE (Smoothed)
9
MMLU
75.25Task Success Rate (TSR)
9
MMLU Math
51.4Score
9
MMLU
91.02Accuracy
9
MMLU STEM
92.22Accuracy
9
MMLU-Pro
46.89Accuracy (maj@2)
9
MMLU Pro NoMath (unseen)
62MMLU Pro (NoMath) Score
9
MMLU-Pro 0-shot
44.4MMLU-Pro Score (0-shot)
9
MMLU-Pro
88.4Accuracy (Knowledge & QA)
9
MMLU-CK (test)
60.95Accuracy
9
MMLU
0.87AUROC
9
MMLU-Pro
72.47Accuracy
9