MMLU-Pro
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-Pro (test)
28.6Accuracy
8
MMLU-Pro Law n=1,101 (test)
75.7Accuracy
8
MMLU-Pro
40.3pass@1
8
MMLU-Pro Nutrition health
0.78Spearman Correlation
8
MMLU-Pro Medical genetics health
0.42Spearman Correlation
8
MMLU-Pro health Human aging
0.62Spearman Correlation
8
MMLU-Pro health Virology
0.55Spearman Correlation
8
MMLU-Pro Psychology
0.983Calibration Threshold (q-hat)
8
MMLU-Pro Health
99.2Calibration Threshold (q-hat)
8
MMLU-Pro Chemistry
0.99Calibration Threshold (q-hat)
8
MMLU-Pro Law
0.997Calibration Threshold (q-hat)
8
MMLU-Pro Zipf 1.4
87.4Accuracy
7
MMLU-Pro Zipf 1.1
81.8Accuracy
7
MMLU Pro Social Sciences (Out-of-Domain)
59.2LPM
7
MMLU Pro Humanities Out-of-Domain
51.74LPM
7
MMLU-Pro (test)
65.3Accuracy
6
MMLU-Pro OOD (test)
80.92Biology Avg@16
6
MMLU-PRO
9.3Best-of-16 Delta
6
MMLU-Pro (test)
56.3Accuracy
6
MMLU-Pro Non-trivial subset All 14 subjects (test)
33.93Accuracy
5
MMLU-Pro OOD
54.7Accuracy (MMLU-Pro OOD Science)
5
MMLU-Pro (test)
89.6Accuracy
5
MMLU-Pro Chemistry
19.5Accuracy
4
MMLU-Pro headline
4Unresolved Pair Count
4
MMLU-Pro Biology (Hindi)
55.6Accuracy
4