MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU
0.94NLL
10
MMLU
61.08Overall Accuracy
10
MMLU Pro
67.84Accuracy
10
MMLU (T1)
48.1Accuracy
10
MMLU-ProX (T1)
26.4Accuracy
10
M-MMLU-c
26.74Normalized Accuracy (M-MMLU-c)
10
MMLU
32.8Accuracy
10
MMLU corrupted ground truth
86.3Accuracy (r)
10
MMLU ProX
86.4Accuracy
10
MMLU-Pro
36.1Pass@1
10
MMLU
61.3Pass@1 Accuracy
10
MMLU
26.7MMLU Reasoning & Comprehension Accuracy
10
MMLU-Pro Debate
65.7Success Rate (SR)
10
MMLU-PRO
92.86Performance
10
MMLU non-IID distribution, alpha=0.1
39.79Accuracy
10
MMLU & MMLU-Pro
77.95Accuracy
10
MMLU-Pro
88.7Math Accuracy
10
MMLU Dialectal Arabic
69.98Accuracy (MMLU DA)
10
MMLU 3-order
55.2Accuracy
10
MMLU 2-order
55.2Accuracy
10
MMLU 1-order
55.2Accuracy
10
MMLU-Pro
25.8TPS
10
MMLU
4.06Tau (τ)
10
MMLU-Redux (unseen categories)
72Accuracy
10
MMLU
4.051LIS
10