MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU Pro OOD official (test)
73.43Overall Accuracy
6
MMLU-pl Global (test)
43.47Accuracy
6
MMLU-fr Global
49.95Accuracy
6
MMLU-ProX-Zh
45.2Accuracy
6
MMLU-Pro
48.4Accuracy
6
MMLU all subjects (test)
61.5Accuracy
6
MMLU
70Accuracy
6
MMLU Redux
92.9Exact Match
6
MMLU
91.8EM
6
MMLU Machine Learning (test)
88.4Accuracy
6
MMLU (Hendrycks Tasks) Part 2
30.7High School Microeconomics Accuracy
6
MMLU-Pro
25.7Law Accuracy
5
MMLU
75.68MMLU Source Lib. Accuracy
5
MMLU
0.92ΔMMLU
5
MMLU-Pro
94.5Sel-AUC
5
MMLU
1,160Latency (NoKV, ms)
5
MMLU
73.5Utility Score
5
MMLU-ProX
60.27Accuracy
5
MMLU Low tier
66.7Reward
5
MMLU High tier
77.4Reward
5
MMLU
3BRR Ratio
5
MMLU STEM → Hum.
4.1ECE
5
MMLU
0.636AUROC (MMLU QA)
5
MMLU
89.8MMLU Biology Score
5
MMLU
87.68Pass@1 Score
5