MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU continuation (test)
39.65Accuracy
12
MMLU
73Mean Accuracy (MA)
12
MMLU
73.89MMLU Score
12
MMLU clinical topics
91.6Accuracy
12
MMLU-X
69.3Accuracy
12
MMLU Redux, Pro (test)
69.7MMLU-R Score
12
MMLU Redux (test)
66.9Accuracy
12
MMLU
67.6MMLU Accuracy (0-shot)
12
MMLU
55.8MMLU Accuracy
12
MMLU
59.69MMLU (5-shot)
11
MMLU-Pro n=8312 (test)
97.6Activation (Act)
11
MMLU
37.31STEM Accuracy (MMLU)
11
MMLU-Sci
39.6Accuracy
11
MMLU CS
38Accuracy
11
MMLU-Pro matched-N groups (test)
65.31Accuracy
11
MMLU-Pro
7.79ECE
11
MMLU-Pro
19.26Brier Score
11
MMLU-Pro (val)
42.9Accuracy
11
MMLU-Pro (val)
38.6Accuracy
11
MMLU LONGGENBENCH
81.8Accuracy (n=15)
11
MMLU-Pro session=13
46.93Average Accuracy (AA)
11
MMLU
50.12Accuracy
11
MMLU-Pro
989.8Avg Tokens
11
MMLU (full)
75.9Macro Accuracy
11
MMLU
79.9MMLU Accuracy
11