MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
Welsh MMLU-Lite
61.25Accuracy
2
MMLU-Pro
4Runtime
2
MMLU 200 samples
0.45MMLU Log-likelihood Score
2
MMLU-Pro
1.134Alpha
2
MMLU Redux
93.3Pass@1
2
MMLU PRO
17.58Accuracy
2
MMLU Global Facts
3Baseline Wins
2
MMLU Pro
57.3Normalized Log Accuracy
2
MMLU Full Text
83Normalized Log Accuracy
2
Full Text MMLU 5-shot
65.2Normalized Log Accuracy
2
MMLU CS (test)
37MMLU CS Accuracy
2
MMLU Non-Target Subjects
26.82mEM
2
MMLU All Subjects
49.57Exact Match (EM)
2
MMLU
625Learned Count
2
MMLU-Pro 5-shot (test)
54.56Throughput
2
MMLU
100ISR
2
MMLU ProX Health (random sample)
50Clarity Score
2
MMLU EN
42.18Knowledge Score
2
MMLU
36.4IoU
2
MMLU-Redux
87.3Mean@1 Accuracy
2
MMLU Finance
50Accuracy
2
MMLU 4-way Zero-shot transfer
91.184Accuracy
2
MMLU-CF
72.5Accuracy
2
MMLU zero-shot
1.03Average Loss
2
MMLU College Physics (test)
95.1Accuracy
2