MMLU
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
MMLU-Pro
77.7Accuracy
11
MMLU-Pro
1,350Token Cost (tokens/question)
11
MMLU
0Cost (cents)
11
MMLU (test)
0Cost (cents)
11
MMLU 14k (test)
1.07MAE
11
MMLU
87.21MMLU Score
11
MMLU Indonesian 1.0 (test)
76Accuracy
11
MMLU Arabic translated (test)
72MMLU
11
MMLU Russian (test)
76Accuracy
11
MMLU Japanese (test)
73Accuracy
11
MMLU Chinese (test)
74MMLU
11
MMLU Portuguese (test)
75Accuracy
11
MMLU Vietnamese (test)
71MMLU Accuracy
11
MMLU
0.772Accuracy
11
MMLU Med
88.1Accuracy
11
MMLU-Pro
90.1Exact Match
11
MMLU 1000
48Pass@1
11
MMLU bio
95.6Accuracy
11
MMLU mimicked
72.5Algebra Score
11
MMLU
54.6Accuracy
10
MMLU STEM
84.91Score
10
MMLU Physics (test)
54.17Accuracy
10
MMLU Chemistry (test)
55.73Accuracy
10
MMLU (val)
77.3Anatomy (AN) Accuracy
10
MMLU
4.37F1
10