MGSM
Benchmarks
TaskDatasetSOTA resultTrendResults
MGSM
93.2Accuracy
245
MGSM (test)
93.4Accuracy
109
MGSM (test)
88Accuracy (ZH)
80
MGSM
88.8Accuracy (Bn)
66
MGSM
94.9Accuracy
64
MGSM
90.79AUC
51
MGSM
84.62AUC
42
MGSM Rev2
99.4Overall Result Accuracy
36
MGSM 1.0 (test)
69.6Accuracy (ru)
35
MGSM
94.4Accuracy
29
MGSM Thai (test)
41.6Accuracy
25
MGSM
84.43Mean@3
23
mGSM v2
78Accuracy (Seen)
21
MGSM
6.08Speedup (de)
20
MGSM non-EU languages (test)
91.4Accuracy
16
MGSM 24 official EU languages
93Accuracy
14
MGSM Bangla
0.88Accuracy (Original)
13
MGSM Rev2 (all)
85.9Accuracy
12
MGSM out-of-domain Rev2
83.6Accuracy
12
MGSM in-domain Rev2
90.4Accuracy
12
MGSM
44Average@10
12
MGSM ALL 1.0 (test)
75.93Accuracy
12
MGSM EN 1.0 (test)
94Accuracy
12
MGSM
72.89Bn Score
12
Bn-MGSM (test)
89.2Accuracy
12