CWQ
Benchmarks
TaskDatasetSOTA resultTrendResults
CWQ
85.2Hit@1
245
CWQ (test)
88.95Hits@1
158
CWQ
81.4Hits@1
84
CWQ (test)
81.3F1 Score
44
CWQ Freebase (test)
86Hits@1
38
CWQ
55.8Pass@1
36
CWQ
86.3Hits@1
30
CWQ
23.62Accuracy
30
CWQ
85.2Hits@1
27
CWQ (test)
92.88Binary Accuracy
24
CWQ
51.74Answer F1
18
CWQ 50% KB
61.4MRR
16
CWQ (30% KB)
58.8MRR
16
CWQ Switzerland
68Accuracy
14
CWQ (test)
82.43Precision
13
CWQ
1# Calls
12
CWQ 50% KB
50.8Hits@1
12
CWQ 30% KB
50.2Hits@1
12
CWQ
87.4F1 Score
11
CWQ
87.4F1 Score
11
CWQ
25Exact Match (EM)
10
CWQ
82.2Hits@1
10
CWQ 40% knowledge gap
39.33EM
7
CWQ multi-entity
64.9F1 Score
7
CWQ (hidden test)
67.1Accuracy
7