Global MMLU
Benchmarks
TaskDatasetSOTA resultTrendResults
Global-MMLU-Lite
58.5Seen Accuracy
21
Global-MMLU Lite 5 Unseen Languages
71.9MMLU Score
20
Global-MMLU-Lite 8 Seen Languages
65.2MMLU Score
20
Global-MMLU Medical
89.1Accuracy (ZH)
17
Global MMLU Lite (subset of 18 languages)
90.13Accuracy
15
Global MMLU Ukrainian (test)
67.03Accuracy (%)
14
Global-MMLU Low
52.31Accuracy
12
Global-MMLU High
68.35Accuracy
12
Global MMLU-Lite Māori
54.64Accuracy
10
Global MMLU 42 languages 1.0 (test)
54.8Average Accuracy
6
Global-MMLU Japanese ja (test)
74AUROC
5
Global-MMLU Russian (test)
75AUROC
5
Global-MMLU Spanish es (test)
74AUROC
5
Global MMLU Overall
59.2Accuracy
5
Global-MMLU Japanese
0.72AUROC
2
Global-MMLU Russian
73AUROC
2
Global-MMLU Polish
77AUROC
2
Global-MMLU Spanish
78AUROC
2
Global-MMLU English
0.75AUROC
2
Global-MMLU French
0.76AUROC
2
Global-MMLU all languages average
0.5AUROC
2
Global MMLU 15 languages
54.77Macro Accuracy
2
Global MMLU (test)
23.46Accuracy (RUS)
2