MMLU-Pro
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-Pro 600 sampled questions
72.27Average @5
4
MMLU-Pro OOD
74.4CPT Score (85%)
4
MMLU-Pro OOD
66.14CPT (80%)
4
MMLU-Pro (test)
79.55Mean Accuracy
4
MMLU-Pro (test)
83.76Accuracy
4
MMLU-Pro English medical subsets
64.7Accuracy
3
MMLU-Pro Health (English)
59.3Accuracy
3
MMLU-Pro law 1.0 (test)
71.05Accuracy
3
MMLU-Pro OpenR1-Math Harder
71.3Accuracy
3
MMLU-Pro (test)
595Optimization Token Usage
3
MMLU-Pro (test)
778Optimization Token Usage (k)
3
MMLU-Pro Overall (test)
0.246Mean Entropy (R1)
3
MMLU-Pro v1 (test)
44.1Accuracy
3
MMLU-Pro (test)
58.84Accuracy (%)
2
MMLU-Pro Adversarial Setting (test)
98.9Accuracy
2
MMLU-Pro
75.2Mean@1
2
MMLU-Pro (test)
43.14Accuracy (%)
1