SuperGLUE
Benchmarks
TaskDatasetSOTA resultTrendResults
SuperGLUE (dev)
93.2Average Score
91
SuperGLUE
91.3SGLUE Score
84
SuperGLUE (test)
92.4BoolQ Accuracy
74
SuperGLUE
94.5CB Accuracy
32
SuperGLUE
76.9WSC Score
25
SuperGLUE
75.9MultiRC Score
22
SuperGLUE
96SST-2 Accuracy
18
SuperGLUE RoBERTa-large (test)
89.21ReCoRD
17
SuperGLUE few-shot
0.818BoolQ Accuracy
16
SuperGLUE 1,000 examples
84BoolQ Accuracy
15
SuperGLUE
83COPA Score
14
SuperGLUE
96.4CB Accuracy
14
SuperGLUE Full, excl. ReCoRD (dev)
70.03Macro Avg Score
13
SuperGLUE 1k samples, excl. ReCoRD (dev)
65.84Macro Avg Score
13
SuperGLUE 100 samples, excl. ReCoRD (dev)
59.88Macro Avg Score
13
SuperGLUE
59.36BoolQ Accuracy
12
SuperGLUE
60.8RTE Accuracy
12
SuperGLUE (test val)
96SST-2 Accuracy
12
SuperGLUE Zero-shot
88BoolQ Accuracy
11
SuperGLUE 1,000 examples (test)
86.7BoolQ
10
SuperGLUE (val)
89.2Average Validation Score
10
SuperGLUE
94.7SST-2 Accuracy
9
SuperGLUE
36Macro Similarity Score
8
SuperGLUE v1 (test)
91.3BoolQ Acc
7
SuperGLUE
88.5BoolQ Accuracy
6