WTQ
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
WTQ
91.25Accuracy
109
WTQ (test)
64.7Denotation Accuracy
62
WTQ
81.4Accuracy
39
WTQ (Evaluation Set)
47Alignment Score
24
WTQ (train)
44Alignment Score
24
WTQ (test)
78.38Accuracy
23
WTQ (held-in)
80.34Accuracy
21
WTQ
60.9Accuracy
21
WTQ
76.3Exact Match (EM)
12
WTQ
71.3Accuracy
12
ROBUT-WTQ Sentence-Level Paraphrase (test)
81.6Accuracy (Pre-perturbation)
10
ROBUT-WTQ Word-Level Paraphrase (test)
85.8Accuracy (Pre-Perturbation)
10
ROBUT-WTQ Column Adding (test)
81.4Accuracy (Pre-Perturbation)
10
ROBUT-WTQ Column Extension (test)
92Accuracy (Pre-perturbation)
10
ROBUT-WTQ Column Order Shuffling (test)
89Accuracy (Pre-perturbation)
10
ROBUT-WTQ Abbreviation Replacement (test)
82.9Accuracy (Pre-perturbation)
10
ROBUT-WTQ Synonym Replacement (test)
83.6Accuracy (Pre-perturbation)
10
WTQ
76Sim Score
9
WTQ (dev)
77.5Accuracy
9
WTQ
44Recall@1 (Base)
8
WTQ (ADVETA-ADD)
44.6Exact Match (EM)
8
WTQ (ADVETA-RPL)
41.8Exact Match (EM)
8
WTQ original (dev)
44.1Exact Match (EM)
8
WTQ (test)
65.4Accuracy
6
WTQ (test)
62.25ROUGE-1
5