Question Answering on AirQA (test)
45.58Single-Hop PerformanceAgentic Hybrid
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Agentic HybridBase Model=GPT-4o-2024-08-062025.09 | 45.58 | 10.53 | 52.13 | 35.56 | 39.61 | 23 | 25.6 | 33.86 | 47.54 | 38.76 | 29.09 | 35.96 | |
| Agentic Text2SQLBase Model=Qwen2.5-72B-Instruct2025.09 | 43.02 | 11.46 | 43.4 | 40.14 | 36.07 | 21.13 | 29.95 | 35.43 | 49.18 | 35.37 | 31.59 | 34.27 | |
| Agentic Text2SQLBase Model=GPT-4o-2024-08-062025.09 | 42.17 | 11.15 | 18.4 | 38.38 | 23.19 | 21.6 | 28.99 | 33.07 | 47.54 | 26.44 | 31.02 | 27.77 | |
| Agentic HybridBase Model=Qwen2.5-72B-Instruct2025.09 | 39.03 | 10.84 | 55.21 | 37.32 | 41.71 | 13.15 | 28.02 | 30.71 | 45.9 | 37.74 | 28.53 | 35.07 | |
| Full-Text w/ CutoffBase Model=Qwen2.5-72B-Instruct2025.09 | 36.18 | 8.98 | 0 | 7.04 | 12.56 | 11.27 | 16.91 | 14.17 | 18.85 | 11.86 | 19.67 | 14.13 | |
| Agentic RAGBase Model=GPT-4o-2024-08-062025.09 | 34.19 | 8.36 | 15.63 | 29.58 | 21.36 | 18.78 | 26.57 | 22.83 | 24.59 | 21.36 | 24.1 | 22.15 | |
| Full-Text w/ CutoffBase Model=GPT-4o-2024-08-062025.09 | 33.9 | 8.05 | 0.69 | 5.99 | 13.53 | 7.51 | 13.53 | 12.6 | 18.03 | 9.94 | 21.05 | 13.16 | |
| Agentic RAGBase Model=Qwen2.5-72B-Instruct2025.09 | 32.76 | 9.6 | 15.63 | 30.28 | 22.06 | 15.96 | 25.12 | 25.98 | 18.85 | 21.02 | 25.21 | 22.23 | |
| RAGBase Model=Qwen2.5-72B-Instruct2025.09 | 31.91 | 7.43 | 18.75 | 21.83 | 22.06 | 11.27 | 19.32 | 20.47 | 21.31 | 19.55 | 21.88 | 20.22 | |
| RAGBase Model=GPT-4o-2024-08-062025.09 | 31.62 | 4.95 | 18.75 | 16.55 | 20.29 | 12.68 | 16.91 | 17.32 | 18.03 | 18.19 | 18.56 | 18.3 | |
| Text2SQLBase Model=Qwen2.5-72B-Instruct2025.09 | 22.22 | 4.02 | 11.11 | 13.38 | 13.85 | 8.45 | 15.46 | 10.24 | 11.48 | 12.43 | 14.13 | 12.92 | |
| Text2SQLBase Model=GPT-4o-2024-08-062025.09 | 21.08 | 6.81 | 7.64 | 17.25 | 14.01 | 8.92 | 12.08 | 16.54 | 14.75 | 11.41 | 18.28 | 13.4 | |
| Title-AbstractBase Model=Qwen2.5-72B-Instruct2025.09 | 17.66 | 6.19 | 0 | 8.1 | 8.05 | 6.1 | 12.08 | 7.87 | 7.38 | 6.44 | 13.3 | 8.43 | |
| Title-AbstractBase Model=GPT-4o-2024-08-062025.09 | 11.4 | 5.26 | 0 | 5.28 | 5.96 | 4.23 | 8.7 | 4.72 | 2.46 | 4.07 | 9.97 | 5.78 | |
| Question OnlyBase Model=Qwen2.5-72B-Instruct2025.09 | 9.69 | 1.86 | 0.35 | 5.99 | 2.74 | 3.29 | 10.63 | 3.94 | 5.74 | 4.52 | 4.99 | 4.65 | |
| Question OnlyBase Model=GPT-4o-2024-08-062025.09 | 8.55 | 1.86 | 1.04 | 5.63 | 4.35 | 1.41 | 10.63 | 2.36 | 0 | 3.95 | 5.54 | 4.41 |