MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU STEM
1,956Response Length (Tokens)
12
MMLU-Pro
61Accuracy
12
MMLU Pro
62.1Accuracy
12
MMLU-Pro
0.84Wall-clock Time (s)
12
MMLU ProX Lite
76.45Macro F1
12
MMLU-Pro
71.54Accuracy (MMLU-Pro)
12
MMLU Pro
0.639WP-AUC
12
MMLU
74.7MMLU Accuracy
12
MMLU 10-subject subset 1.0 (test)
85.4Average Accuracy
12
MMLU SAGE-generated (test)
94.12Overall Accuracy (OA)
12
MMLU Pro
68.21Avg@k
12
MMLU
71MMLU Accuracy
12
M_MMLU_c
30.07Accuracy (MMLU-c)
12
MMLU
71.6Accuracy
12
MMLU
66.5Accuracy
12
MMLU
66.5Accuracy (MMLU)
12
MMLU Pro
47.12Accuracy
12
MMLU-Pro
66.82Accuracy
12
MMLU-Pro (test)
24.5Accuracy
12
MMLU (test)
47.9Accuracy
12
MMLU-pro
49.3Accuracy
12
MMLU CF
100Perturbation Success Rate
12
MMLU
73.5Accuracy
12
MMLU Hindi
67Accuracy
12
MMLU Korean (test)
72Accuracy
12