MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMLU German (test)
73Accuracy
15
MMLU
79.95Zero-shot Accuracy
14
MMLU
63.34Accuracy
14
MMLU
59.7EN Accuracy
14
MMLU-Pro
81.5Accuracy (MMLU-Pro)
14
MMLU
71.08Accuracy
14
MMLU
80.15Accuracy (MMLU)
14
Global-MMLU (test)
0.78AUROC
14
MMLU-Pro (held-out test)
83.67Accuracy
14
MMLU (test)
90.84Accuracy
14
MMLU-Pro
80Pass@1
14
MMLU-Pro
49.54ACC
14
MMLU
65.1Accuracy
14
MMLU-Pro 233-sample stratified
86.71Accuracy
14
MMLU
68.71STEM Accuracy
14
MMLU
53.8STEM Accuracy
14
MMLU
81.6PRR Accuracy
14
MMLU-Pro stratified sampling: 150 samples per category
54.78Accuracy
14
MMLU stratified sampling 50 samples per category
79.18Accuracy
14
MMLU-ProX
82.6Average Accuracy
14
Global-MMLU
90.5Accuracy (ZH)
14
MMLU
92MMLU Accuracy
14
MMLU
91.9MMLU Anatomy Accuracy
14
MMLU-CF
0.98Relative Cost
14
MMLU-Pro
61.6Average@16 Accuracy
14