MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-EE (test)
68.97Accuracy
7
MMLU Redux
77.5Business Ethics Accuracy
7
MMLU
81.8Normalized Log Accuracy (MMLU)
7
MMLU Law (Far-Domain)
83.4AUROC
7
MMLU
68.5Accuracy
7
MMLU
78.5MMLU Score
7
MMLU
0.021MAE
7
MMLU-CF first 1,000 samples (test)
74.2Exact Match Accuracy
7
MMLU
46.21Humanities Score
7
MMLU Pro Persian
24.1Accuracy
7
MMLU
90.1EM (World Knowledge)
7
MMLU 5-shot
75.14Accuracy (5-shot)
7
MMLU Pro Others (Out-of-Domain)
0.624LPM
7
MMLU Pro STEM (Out-of-Domain)
59.2LPM
7
MMLU In-Domain
78.51LPM
7
MMLU Pro Out-of-Domain
65.32STEM Score
7
MMLU In-Domain
0.679AUROC
7
MMLU-Redux
90.89Exact Match
7
MMLU Pro
33.17MMLU Pro Score: Computer Science
7
MMLU Expert (test)
0.757Astronomy Accuracy
7
MMLU STEM
63.9Accuracy
7
MMLU-Pro (eval)
117.19MAE
6
MMLU Pro STEM
0.897Train R-squared
6
MMLU (test)
84.6Accuracy
6
MMLU (test)
74.36Pooled Accuracy
6