MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-Redux 2.0
97.77Pass@1 Accuracy
8
MMLU-Pro
90.8Pass@1 Accuracy
8
MMLU-Pro Anatomy health
0.62Spearman Correlation
8
MMLU
77.5MMLU Accuracy
8
MMLU-Pro
63.8MMLU-Pro Score
8
MMLU
89.9Average Accuracy (MMLU)
8
MMLU-Pro
75.25Accuracy
8
MMLU Out-of-domain
0.77PRR (Accuracy)
8
MMLU
69Accuracy
8
MMLU
45.5Average Accuracy
8
MMLU
92Normalized Score
8
MMLU Elementary Math
34.95Recall
8
MMLU Computer Security
46Accuracy
8
MMLU Corporate Biology
60.4Accuracy
8
MMLU Perturbed
53.5Accuracy
8
MMLU-Pro Economics
0.98Calibration Threshold (q-hat)
8
MMLU-Pro Math
0.126Calibration Threshold (q-hat)
8
MMLU-Pro Physics
0.973Calibration Threshold (q-hat)
8
MMLU-Pro Engineering
0.995Calibration Threshold (q-hat)
8
MMLU-Pro
47.5Accuracy
8
MMLU Clinical Knowledge (test)
63.21Accuracy
8
MMLU Pro STEM
54.73Science Knowledge Score (MMLU Pro STEM)
8
MMLU Pro
78.57Accuracy (Acc)
8
MMLU 0-shot
69.11Exact Match (EM)
8
MMLU
77.84Accuracy
8