RACE
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
RACE (test)
2.18Estimated Difficulty
15
RACE
23.29EM
14
Race 3shots
83.31Accuracy
14
RACE MRQA out-of-domain
43.62EM
14
RACE
83.36Vanilla Accuracy
14
RACE Full (test)
53.3Accuracy
13
Race Japanese
0.64Alignment
12
Race English
65Alignment Score
12
RACE
31.39Normalized Accuracy
11
RACE (test)
59.9Accuracy
11
RACE++
68.3Accuracy
11
RACE
83.08Exact Match
11
RACE Audio
64.21Accuracy (Sub-task 1)
10
race
86.51Accuracy
10
RACE high
84.6Accuracy
10
RACE standard (test)
66.4Accuracy
10
RACE-h (val)
84.3Accuracy
9
RACE-m (val)
86.8Accuracy
9
RACE (dev)
89.5Accuracy
9
RACE M
57.17Accuracy (RACE M)
8
RACE
89.69Accuracy (ACC)
8
RACE Generalization
35.6Accuracy
8
RACE (dev)
88.1Accuracy
8
RACE-m (test)
75.1Accuracy
7
RACE (test)
2.3Speedup
7