MMMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
MMMLU (test)
66.88Accuracy
52
MMMLU
76.1CLCall
30
MMMLU
82.25MMMLU General Knowledge Accuracy
29
MMMLU Danish (test)
46.24MMLU Score
25
MMMLU (Massive Multilingual Language Understanding)
79.5Accuracy
21
MMMLU
60.5Accuracy (Korean)
20
MMMLU
87.2Accuracy
18
MMMLU Swahili 1.0 (test)
33.38Accuracy
18
MMMLU Korean 1.0 (test)
41.94Accuracy
18
MMMLU non-EU languages (test)
77.4Accuracy
16
MMMLU 24 official EU languages
80.6Overall Score
14
MMMLU
76.1CLCall Score
10
MMMLU
91.8MMMLU Score
9
MMMLU Marathi (test)
34.6MMMLU Accuracy
9
MMMLU ko, de, es, ja
88.9Average Score
9
MMMLU
37.08MMMLU Score
8
MMMLU
36.14Accuracy
8
MMMLU PT_BR
86.6Accuracy
5
MMMLU (target)
57.86RPR
5
MMMLU (target)
99.49RPR
5
MMMLU (target)
55.14Accuracy
5
MMMLU German
60.8Normalized Log Accuracy
4
MMMLU
90.3EM (Exact Match)
3
MMMLU German 5-shot (test)
61.8Normalized Log Accuracy
3
MMMLU 5-shot
78.94Accuracy
3