RTE
Benchmarks
TaskDatasetSOTA resultTrendResults
RTE
93.5Accuracy
590
RTE
84.84Accuracy
110
RTE
83.13Accuracy
78
RTE (test)
90.25Accuracy
52
RTE
84.8Accuracy (0-shot)
42
RTE (test)
76.53Accuracy
26
RTE (val)
0.918Accuracy
24
RTE
26.24Delta 1
24
RTE
81.2Avg Accuracy
21
RTE (Recognizing Textual Entailment) GLUE (val)
66.06Accuracy
18
RTE
62.82Zero-shot Accuracy (RTE)
17
RTE Neg
88.1Accuracy (RTE Neg)
14
RTE
94.4Normalized Accuracy
13
RTE
87Utility Accuracy
12
RTE (dev)
90.5Accuracy
12
RTE
79Accuracy (RTE)
9
RTE
4.29Total Communication Time ($10^3$ s)
9
RTE
100Repair Accuracy
8
RTE GLUE (test dev)
84Accuracy
8
RTE SuperGLUE (test)
66.13Accuracy
8
RTE
80.91F1 Score
7
RTE
65.8Macro-F1
6
RTE
92.1F1 Macro
5
RTE
1.29Wall-clock Time (hours)
4
RTE low-data regime GLUE
58.53Accuracy
4