MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU high-perf.
78.12MMLU* Accuracy (high-perf.)
7
MMLU
73.040-shot Accuracy
7
MMLU
1Spearman Rho
7
MMLU
59.01Accuracy
7
MMLU
82.75Accuracy (HS Chem)
7
MMLU Professional Law (test)
80.78Accuracy
7
MMLU HS World History (test)
95.38Accuracy
7
MMLU HS US History (test)
94.26Accuracy
7
MMLU College Medicine (test)
87.88Accuracy
7
MMLU Professional Law (test)
89.52Accuracy
7
MMLU HS World History (test)
93.14Accuracy
7
MMLU HS US History (test)
93.93Accuracy
7
MMLU College Medicine (test)
81.92Accuracy
7
MMLU Pro
82.6MMLU Pro Score
7
MMLU stratified subset (test)
0.8AUROC
7
MMLU
73.21Accuracy
7
MMLU-Pro CoT
71.8Health Accuracy
7
M MMLU_c
29.87Accuracy (MMLU_c)
7
M_MMLU_c Spanish
31.37Acc (Normalized)
7
MMLU-Pro
5.31MMLU-Pro Delta (%)
7
MMLU STEM
79.05Pass@1 (MMLU STEM Reasoning)
7
MMLU OOD
86.6Accuracy
7
MMLU-Pro
80Accuracy (Acc)
7
MMLU
83.3Accuracy
7
MMLU-Pro Zipf 0.8
76.7Accuracy
7