MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU
2.07Average Inference Time (s)
30
MMLU STEM
74.1Accuracy
30
MMLU-CF
71.6EM
30
MMLU-redux
90.6Accuracy
30
MMLU
100Accuracy
30
MMLU-PRO
81.61Accuracy
29
MMLU
72.45MMLU Accuracy
29
MMLU-M
27.4Accuracy
29
MMLU CoT
72.8MMLU (CoT)
29
MMLU-P
97.1Accuracy
29
MMLU
90.8During-task Accuracy
29
MMLU-Pro
42.48EM Score
28
MMLU-Pro
73.44Pass@1
28
MMLU-ProX
79.5Accuracy
28
MMLU Moral Scenarios (test)
0.86Accuracy
28
MMLU Pro
84Accuracy
27
MMLU
79.45Accuracy
27
MMLU
73.1Accuracy
27
MMLU-Pro
75.41Accuracy (MMLU-Pro)
27
MMLU
89.5Accuracy
27
MMLU zero-shot (test)
76Accuracy (MMLU zero-shot)
27
MMLU
92Detection Accuracy
27
MMLU-Pro
71Pass@1
27
MMLU-Pro
84MMLU-Pro General Knowledge EM
27
MMLU-M
28.79Accuracy
26