MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-Pro
82.93Accuracy
23
MMLU
89.1Accuracy
23
MMLU STEM
77.8Accuracy
23
MMLU-Pro
58.8Accuracy
22
MMLU Formal Logic (test)
73.67Accuracy
22
MMLU
78Accuracy (pass@1)
22
MMLU
62.5PRR
22
MMLU
89.3PRR
22
MMLU-Pro
79.7MMLU-Pro Score
22
MMLU
92.39Accuracy
22
MMLU
59.37MMLU
22
MMLU
74.5Accuracy
22
MMLU STEM (test)
82.8Accuracy
22
MMLU-Pro
2.44Normalized Speedup
21
MMLU
73.1Accuracy
21
MMLU
0.087Error Rate (ER)
21
MMLU
2.5ECE
21
MMLU-Pro
35.88maj@4 Accuracy
21
MMLU
62.38Accuracy
21
TinyMMLU
86.8Accuracy
21
MMLU
65.81MMLU Score (x100)
21
MMLU
92.2p_T Score
21
MMLU
83.7Accuracy
21
MMLU CSec
88CSec Score
21
MMLU en
70.23MMLU (en)
21