MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU-Pro
55Average Score
66
MMLU Pro
96.8Accuracy
66
MMLU CF
74.5Score
66
MMLU
82.03MMLU Accuracy
64
MMLU Pro
82.93Accuracy
64
MMLU-Pro
82.05Accuracy
64
MMLU-Pro
92.85Pass@1
64
MMLU-CF
75.9Accuracy
64
MMLU-Pro
90.1MMLU-Pro General Reasoning Avg@8 Acc
63
MMLU
87.48AUPRC
62
MMLU-Pro
94.21Average Score
60
MMLU-Pro
72.8MMLU-Pro Accuracy
60
MMLU (full)
88.3MMLU Accuracy
60
MMLU
0.056Brier Score
58
MMLU
91.35Accuracy
57
MMLU-Redux
92.91Accuracy
57
MMLU-Pro
82MMLU-Pro General Knowledge Score
56
MMLU-Pro Natural Setting (test)
87.8Accuracy
56
MMLU
84.2MMLU Accuracy
55
MMLU
0.141Test Error Probability
52
MMLU
74.91MMLU Accuracy (Overall)
52
MMLU
64.07AUC
51
MMLU
86.21Accuracy
51
MMLU
84F1 Score
50
MMLU
0FPR (MMLU)
50