Knowledge Base Question Answering on GrailQAbility unanswerable 1.0 (test)
94.21F1 (L)RetinaQA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| RetinaQATrain=A+U, Thresholding=true2024.03 | 94.21 | 92.38 | 82.1 | |
| TIARATrain=A+U, Thresholding=true2024.03 | 93.43 | 91.24 | 66.65 | |
| TIARATrain=A+U, Thresholding=false2024.03 | 92.4 | 89.82 | 62.24 | |
| RnG-KBQATrain=A+U, Thresholding=true2024.03 | 92 | 89.8 | 67.2 | |
| RetinaQATrain=A, Thresholding=true2024.03 | 90.02 | 86.96 | 71.2 | |
| RnG-KBQATrain=A+U, Thresholding=false2024.03 | 89.7 | 86.4 | 59.4 | |
| PanguTrain=A, Thresholding=true2024.03 | 86.48 | 82.7 | 54.68 | |
| RetinaQA - LFRTrain=A+U, Thresholding=true, Ablation=minus logical form retriever2024.03 | 85.38 | 82.43 | 54.17 | |
| RetinaQA - LFITrain=A+U, Thresholding=true, Ablation=minus logical form integrator2024.03 | 83.01 | 78.95 | 25.13 | |
| PanguTrain=A+U, Thresholding=true2024.03 | 80.46 | 80.32 | 55.03 | |
| TIARATrain=A, Thresholding=true2024.03 | 71.6 | 66.68 | 24.15 | |
| RetinaQATrain=A+U, Thresholding=false2024.03 | 69.87 | 65.04 | 40.14 | |
| RetinaQATrain=A, Thresholding=false2024.03 | 67.93 | 63.16 | 38.45 | |
| TIARATrain=A, Thresholding=false2024.03 | 64.95 | 58.31 | 9.2 | |
| RnG-KBQATrain=A, Thresholding=true2024.03 | 59.9 | 54.5 | 33.6 | |
| RetinaQA - (SG U SR)Train=A+U, Thresholding=true, Ablation=minus sketch generator and schema retriever together2024.03 | 47.84 | 41.54 | 20.7 | |
| RnG-KBQATrain=A, Thresholding=false2024.03 | 46.9 | 40.1 | 5.7 | |
| PanguTrain=A+U, Thresholding=false2024.03 | 31.42 | 21.89 | 13.85 | |
| PanguTrain=A, Thresholding=false2024.03 | 31.4 | 22.25 | 15.13 |