MMLU-Pro
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU-PRO
50.72Average Accuracy (AA)
56
MMLU-PRO zero-shot
84.29Accuracy
51
MMLU-Pro Science
69.1M-F1
40
MMLU-Pro Chem.
72.8Accuracy
40
MMLU-Pro 5-shot
44.65Knowledge Score (5-shot)
37
MMLU-Pro Health (FR) X (test)
66.08Accuracy
35
MMLU-Pro Health
84Accuracy
28
MMLU-Pro Math
89.79Accuracy
26
MMLU-Pro
71.4Best Accuracy
25
MMLU-Pro Law (test)
12.1HALL%
21
MMLU-Pro Engineering
81Accuracy
20
MMLU-Pro (test)
41.9Accuracy
18
MMLU-Pro
50.7Pass@1
15
MMLU-Pro
76.12Pass@3
14
MMLU-PRO (test)
56.37Pass@1 Accuracy
13
MMLU-Pro n=8312 (test)
100Activation
11
MMLU-Pro (test)
23.4Accuracy (MMLU-Pro Test)
11
MMLU-Pro Biology English
77.7Accuracy
11
MMLU-Pro (test)
23.6MMLU-Pro (test) Accuracy
11
MMLU-Pro AceReason (Reduced)
71.1Accuracy
10
MMLU-Pro AceReason (Complete)
76.5Accuracy (MMLU-Pro AceReason)
10
MMLU-Pro 80 (test)
38.68Pass@1
10
MMLU-Pro (test)
37.72Accuracy
10
MMLU-Pro
0.037Brier Score
9
MMLU-Pro
91.6BCA
9