MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU o=1 (Semantic-level)
76.9Accuracy
21
MMLU Semantic-level split, o=3
90.1Accuracy
21
MMLU Pro
68.08Pass@1
21
MMLU Pro (test)
73.21Overall Accuracy
21
MMLU (test)
86.4Accuracy
21
MMLU professional medicine
94Accuracy
21
MMLU-Pro Physics
88.5Accuracy
20
MMLU College
73.12Accuracy (MMLU College GK Reasoning)
20
MMLU High
67.2Accuracy
20
MMLU PRO
0.711Accuracy (ACC)
20
MMLU-ProX Lite
83Target Accuracy
20
MMLU-Pro
266.5AUPC
20
MMLU
65MMLU Performance (MM)
20
MMLU-Pro Computer Science
63.2Accuracy
20
MMLU-Pro CS
56.8Performance
20
MMLU-Pro
44.8MMLU-Pro Chemistry Accuracy
20
MMLU
74Answer-letter Accuracy
20
MMLU 16 subjects filtered
97.11ASR@30
20
MMLU Pro
67.8Overall Score
20
MMLU (test)
57.83BS Score
20
MMLU-Pro
0.678Overall Score
20
MMLU
64.31MMLU Accuracy
20
MMLU Pro
82.8Biology Accuracy
20
MMLU
75.9PRR
20
MMLU Pro
76.4Accuracy
20