RACE
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
RACE high
94.5Accuracy
295
RACE mid
89.9Accuracy
196
RACE
89.95Accuracy
151
RACE (test)
95.4RACE Accuracy (Medium)
111
RACE
68.3Accuracy
86
RACE
74.93Accuracy
75
RACE
98.24Accuracy
64
RACE to MMLU
87.57AUROC
41
RACE
94.5RACE Overall Accuracy
38
RACE-m
0.931Accuracy
31
RACE
93.3Accuracy (RACE)
26
RACE-h
62.3Accuracy
26
RACE-h (test)
89.95Accuracy
26
RACE Llama-3.1-8B and Gemma-2-9B backbones (test)
91.3AUROC
24
RACE
87.3First-Token Accuracy
24
RACE MRQA out-of-domain evaluation
46.3EM
23
RACE
70.2RACE Middle Score
21
Race Even
0.072MAE
20
RACE Middle
67.27Score
20
RACE-C
93.66Accuracy
19
Race M
45.47Race M Score
18
Race-H
38.56RACE-h Score
18
RACE Middle School
95.4Accuracy (RACE MS)
16
RACE (dev)
88.1Accuracy
16
RACE
91Score
15