MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-Redux
67Accuracy
9
MMLU-R English
75Accuracy
9
MMLU
36.28MMLU Humanities Accuracy
9
MMLU High school US history (test)
93.84Top-1 Accuracy
9
MMLU 16 subjects filtered subset
81.24ASR@30
9
MMLU-Redux
4.22FDR (%)
9
MMLU
7FDR
9
MMLU-Pro
0.771Score
9
MMLU-Redux
79.8Accuracy
9
MMLU-Pro
16.5Personalization Bias (PB)
9
MMLU Health Large Shift (test)
50.6Accuracy
9
MMLU Law Large Shift (test)
37.9Accuracy
9
MMLU English Large Shift (test)
44.4Accuracy
9
MMLU Computer Science Large Shift (test)
46.3Accuracy
9
m-MMLU Non-English
63.84Accuracy
9
MMLU Physics Larger Dist. Shift 37, 36
32.64Accuracy
9
MMLU Chemistry Larger Dist. Shift 37, 36
44.79ACC
9
MMLU
6.2MMLU Δ (pts)
8
MMLU
64Accuracy
8
MMLU-Pro Xmath
75.1FR slc&acc
8
MMLU Pro Xmath
79.9FR SLC&Acc
8
MMLU
85.7Best-of-K Accuracy
8
MMLU-Pro naive task stream
91Accuracy (Economics)
8
MMLU Astronomy
4.3Per-query Runtime (s)
8
MMLU math part
70.6Expert Count
8