MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU
17.3TPS
3
MMLU ProX Lite
79Accuracy (en)
3
MMLU ProX-Lite
78.2Accuracy (en)
3
MMLU
56.7Pass@1
3
MMLU Pro
38.6Normalized Log Accuracy
3
MMLU Full Text
64.9Normalized Log Accuracy
3
MMLU
67Normalized Log Accuracy
3
MMLU Hungarian
44.86Accuracy
3
MMLU Turkish
43.51Accuracy
3
MMLU medical diabetes
74.7Accuracy
3
MMLU-Pro
0.9STS (Act)
3
MMLU
0.15Cost ($)
3
MMLU STEM (test)
0.012Loss
3
Global-MMLU Medical 13 non-English languages
68.2Non-Eng AVG Original Accuracy (0-shot)
3
MMLU
46Error Rate (%)
3
MMLU
4.46Average Relative Improvement
3
MMLU style (test)
51.06Accuracy
3
MMLU-Pro
48.2Baseline Accuracy
3
MMLU
71.3Baseline Acc
3
MMLU Multilingual translated (test)
90.4Accuracy (Arabic)
3
MMLU f_clean o=5
67Acc (Exact)
3
MMLU (test)
76.7Clinical Accuracy
3
MMLU-Pro non-STEM
71AUDC
3
MMLU-Redux 5-shot
92.19Accuracy
3
MMLU-Pro 5-shot
75.58Accuracy
3