MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU
74Pass@1
6
MMLU
37.4MMLU General Knowledge Drop in Utility (%)
6
MMLU
-17.6Drop in Utility
6
MMLU
0.968AUPR (UE)
6
MMLU psychol
46.42Accuracy (MMLU Psychology)
6
MMLU
62Accuracy (General)
6
MMLU Safety Alignment
44.67Change in RA (%)
6
MMLU Stem
82.39MMLU Stem GK Accuracy
6
MMLU
90.3Accuracy (Long-CoT MMLU)
6
MMLU Pro
56.82MMLU-Pro Score
6
MMLU-Pro
0.8R-squared
6
MMLU High school US history (val)
40.91Top-1 Accuracy
6
MMLU 57 subjects (test)
74.3Accuracy
6
MMLU
70.41MMLU Score
6
MMLU
22Attention Time (ms)
6
MMLU-ProX stratified subset 1,000 instances (test)
18.7Accuracy
6
MMLU
70.7Solution Rate
6
MMLU LLaMA 1B
0.001vNMSE
6
MMLU
58MMLU Accuracy Gain (Absolute)
6
MMLU-Pro
0.865MMLU-Pro Accuracy
6
MMLU 1/10th subset (test)
4End-to-End Runtime (hours)
6
MMLU-Pro
65.6Acc (Mean)
6
MMLU Stem (val)
58.13Accuracy
6
MMLU
89.9Pass@1
6
MMLU-Pro
73.5EM (World Knowledge)
6