MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU Pro (test)
68.5EM
5
MMLU in-distribution
74.4Accuracy
5
MMLU-pro
43Performance
5
MMLU L-M pair
74Accuracy
5
MMLU M-Q pair
78Accuracy
5
MMLU L-Q pair
79Accuracy
5
MMLU-Pro
58.4EM
5
MMLU
81.04EM
5
MMLU (test)
95.4Clinical
5
MMLU redux
54.64Accuracy
5
MMLU and MMLU-Pro on LLM Workloads (GPT-2 Large, LLaMA 3.1 8B, DeepSeek-V2 7B)
91.7Coverage
5
MMLU-Pro
64.4MMLU-Pro English Score
5
MMLU STEM (test)
75Maj@k
5
MMLU Chemistry
63Accuracy (MMLU Chemistry Domain Reasoning)
4
MMLU-Pro Computer Science
25.6MMLU-Pro CS Domain Reasoning Accuracy
4
MMLU CS (Computer Science)
70.1MMLU CS Domain Reasoning Accuracy
4
MMLU-Pro Biomedical
53.2Accuracy (MMLU-Pro Biomedical)
4
MMLU Biomedical
70.2Accuracy
4
MMLU last 25% (test)
72Accuracy
4
MMLU-Pro
87.8Score
4
MMLU
22.95MMLU Accuracy
4
MMLU Pro
68.41Utility Score
4
MMLU
83.01MMLU Accuracy (Vanilla)
4
MMLU-Pro
51.8Correlation (%)
4
MMLU Medical (test)
55.91Accuracy
4