MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-Pro
17.47LLMcritic Calls
5
MMLU 50/50 calibration (test)
73.7CR (%)
5
MMLU
91.8Accuracy (pass@1)
5
MMLU
89Arabic Accuracy
5
MMLU
100Accuracy under Attack
5
Global MMLU (T5)
59.1Accuracy
5
MMLU-ProX Overall
43.1Accuracy
5
MMLU T5
50.8Accuracy (MMLU T5)
5
MMLU T1
52.3Accuracy
5
MMLU-ProX T5
25.2Accuracy
5
MMLU-ProX (T1)
23.5Accuracy
5
MMLU-ProX
27.7Accuracy
5
MMLU-Pro
9.1Accuracy Improvement
5
MMLU
49.2MMLU Accuracy (Chemistry)
5
MMLU
6Delta (%)
5
MMLU
0.94Correlation
5
MMLU ProX
81pass@1
5
MMLU
76.89Accuracy
5
MMLU STEM
49.7Accuracy
5
MMLU
10.62Avg Length
5
MMLU
28.1STEM Score
5
MMLU Pro
70.93pass@1
5
MMLU OOD
43.36Accuracy
5
MMLU
0.75False Positive Rate (FPR)
5
MMLU
73.04Accuracy
5