MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU-Pro
80.6Accuracy
130
MMLU-Pro
88.3Accuracy (MMLU-Pro)
122
MMLU 0-shot
70.46Accuracy
119
MMLU
86.4MMLU Score
116
MMLU-Pro
73.44pass@1 Accuracy
115
MMLU Med
92.1Accuracy
111
MMLU (test)
90.46Normalized Accuracy
107
MMLU-Pro
89.1Accuracy
103
MMLU
73.02MMLU Score
98
MMLU-Pro
80.4Score
98
MMLU (val)
74.12Accuracy
94
MMLU
76.5Accuracy
92
MMLU
12ASR@3
91
MMLU
78.95Accuracy
90
MMLU
80.2MMLU Score
86
MMLU
79.6MMLU Accuracy
74
MMLU
88.7Accuracy
74
MMLU
92.16MMLU Knowledge Reasoning Accuracy
73
MMLU 5-shot
73.4Accuracy
73
MMLU (test)
87.6Accuracy
71
MMLU
88.6MMLU Score
70
MMLU
80.75MMLU Multi-Domain Knowledge Acc
68
MMLU-Pro
86.7Accuracy
67
MMLU-Pro
74.04Average Score @1
67
MMLU
0.947ROC-AUC
66