Knowledge Base Question Answering on GrailQAbility answerable 1.0 (test)
81.22F1 (L)RetinaQA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RetinaQATrain=A, Thresholding=false2024.03 | 81.22 | 81.2 | 77.41 | |
| RetinaQATrain=A+U, Thresholding=false2024.03 | 80.98 | 80.97 | 76.95 | |
| TIARATrain=A, Thresholding=false2024.03 | 80.03 | 80 | 75.63 | |
| PanguTrain=A+U, Thresholding=false2024.03 | 79.45 | 79.42 | 73.49 | |
| PanguTrain=A, Thresholding=false2024.03 | 78.72 | 78.7 | 74 | |
| RnG-KBQATrain=A, Thresholding=false2024.03 | 78.1 | 78.1 | 74.2 | |
| RetinaQATrain=A+U, Thresholding=true2024.03 | 77.91 | 77.91 | 75.16 | |
| RetinaQATrain=A, Thresholding=true2024.03 | 77.74 | 77.74 | 75.01 | |
| PanguTrain=A+U2024.03 | 77.43 | 77.43 | 69.52 | |
| RnG-KBQATrain=A+U, Thresholding=false2024.03 | 75.9 | 75.9 | 72.6 | |
| PanguTrain=A, Thresholding=true2024.03 | 75.52 | 75.51 | 72.37 | |
| PanguTrain=A+U, Thresholding=true2024.03 | 75.25 | 75.22 | 71.62 | |
| TIARATrain=A, Thresholding=true2024.03 | 74.08 | 74.05 | 70.56 | |
| RetinaQATrain=A2024.03 | 73.92 | 73.92 | 67.48 | |
| RetinaQA - LFRTrain=A+U, Thresholding=true, Ablation=minus logical form retriever2024.03 | 73.4 | 73.39 | 70.9 | |
| RetinaQA - (SG U SR)Train=A+U, Thresholding=true, Ablation=minus sketch generator and schema retriever together2024.03 | 72.99 | 72.95 | 68.13 | |
| TIARATrain=A2024.03 | 72.02 | 72.02 | 64.93 | |
| RetinaQATrain=A+U2024.03 | 71.69 | 71.69 | 65.55 | |
| PanguTrain=A2024.03 | 71.63 | 71.63 | 65.95 | |
| RnG-KBQATrain=A, Thresholding=true2024.03 | 71.4 | 71.3 | 68.5 | |
| TIARATrain=A+U, Thresholding=false2024.03 | 71.33 | 71.32 | 68.29 | |
| RnG-KBQATrain=A+U, Thresholding=true2024.03 | 70.9 | 70.8 | 68.1 | |
| RetinaQA - LFITrain=A+U, Thresholding=true, Ablation=minus logical form integrator2024.03 | 70.89 | 70.85 | 68.07 | |
| TIARATrain=A+U, Thresholding=true2024.03 | 69.89 | 69.88 | 66.98 | |
| RnG-KBQATrain=A2024.03 | 65.9 | 65.9 | 60.2 | |
| RnG-KBQATrain=A+U2024.03 | 65.8 | 65.8 | 60.8 | |
| TIARATrain=A+U2024.03 | 65.16 | 65.16 | 59.84 |