MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU Pro
87.98AUROC
20
MMLU
52.15Accuracy
20
MMLU-Pro
13.67TPT
20
MMLU
84.9Accuracy
20
MMLU-Pro
66.46pass@1
20
MMLU (test)
0.842MAE (%)
20
MMLU_Pro (test)
0.924Spearman Rho
20
MMLU
55MMLU Score
20
MMLU
75.8EM
20
MMLU-Pro (test)
58.6Accuracy
19
MMLU Adversarially Perturbed
43.5AuA (GCG)
19
MMLU (test)
81.97MMLU Problem Solving Accuracy
19
MMLU
78.33MMLU 5-shot Accuracy
18
MMLU
78.825-shot Accuracy
18
MMLU
68.2Accuracy
18
MMLU-Pro
49.8Accuracy
18
MMLU Med
83.8Performance
18
MMLU out-of-distribution
5.6ECE
18
MMLU-Pro
54.88Accuracy (maj@2)
18
MMLU-Pro
69.4Accuracy
18
MMLU
83.7MMLU AUROC
18
MMLU
65.7vbal Score
18
MMLU
78.2Accuracy
18
MMLU Pro
80.7Accuracy
18
MMLU-Redux
0.108Brier Score
18