MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU Professional Law
87.7Accuracy
26
MMLU
80.515-shot Accuracy
25
MMLU
96.2MMLU General Reasoning Accuracy
25
MMLU
90.8MMLU Score
25
MMLU (test)
80.48Accuracy
25
MMLU
90.1Score
25
MMLU Redux
80.3Accuracy
25
MMLU 153-question shuffled subset
87.28Accuracy
24
MMLU
53.94MMLU Score
24
MMLU
88.21Score
24
MMLU-pro (test)
77.74AUROC
24
MMLU-Pro
68.6Overall Accuracy (MMLU-Pro)
24
MMLU-Pro
73.44Pass@1
24
MMLU-Pro
2,993Output Length
24
MMLU-ProX (test)
43.6Accuracy
24
MMLU (test)
75.3Accuracy
24
MMLU
79.7MMLU First-Token Accuracy
24
MMLU-Pro
3.3Latency (s)
24
MMLU-Pro
31.42Performance
24
MMLU-P
75.6Accuracy
24
MMLU
71.9pass@1
24
MMLU Medical and Biological Sub-tasks
95.8Clinical Knowledge Accuracy
24
MMLU Mathematics (test)
57.69Average Accuracy
24
MMLU-Pro
88.6Biology Score
23
MMLU
83.1Accuracy
23