Knowledge Base Question Answering on CWQ (Hits@1 and F1)
86.3Hits@1STAR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| STARSetting=Lightweight Retriever, RT=1.22026.04 | 86.3 | 69.1 | |
| RD-PSetting=Lightweight Retriever, RT=1.12026.04 | 83.8 | 66.2 | |
| RoGSetting=LLM-centric Retriever, RT=1.22026.04 | 83.3 | 64.8 | |
| Logits-to-LogicModel Scale Category=13b model, Backbone=LLaMA3-13b2025.11 | 83.1 | 64.1 | |
| Logits-to-LogicModel Scale Category=7-8b model, Backbone=LLaMA3-8b2025.11 | 80.9 | 45 | |
| DeSQBackbone=Phi-42026.05 | 78 | — | |
| ChatKBQA2026.05 | 76 | — | |
| GCRModel Category=Hybrid System, Backbone=LLaMA3-8b+GPT4o-mini2025.11 | 75.8 | 61.7 | |
| GCRModel Category=Hybrid System, Backbone=LLaMA3-8b+ChatGPT2025.11 | 72.7 | 60.9 | |
| RoGModel Scale Category=13b model, Backbone=LLaMA3-13b2025.11 | 68.3 | 63.2 | |
| Graph CoTSetting=LLM-centric Retriever, RT=513.62026.04 | 66.1 | 61.4 | |
| RoGModel Scale Category=7-8b model, Backbone=LLaMA2-Chat-7B2025.11 | 62.6 | 56.2 | |
| GNN-RAGModel Category=Hybrid System, Backbone=LLaMA2-7b+GNN2025.11 | 61.7 | 59.4 | |
| KD-CoTModel Scale Category=7-8b model, Backbone=LLaMA2-7b2025.11 | 55.7 | — | |
| DeepSeek-R1-70BSetting=LLM without KG, RT=17.22026.04 | 55.3 | 50.3 | |
| ToGSetting=LLM-centric Retriever, RT=90.42026.04 | 52.1 | 43.1 | |
| ChatGPT+CoTModel Category=Closed-source business model2025.11 | 47.5 | 31 | |
| SubgraphRAGModel Scale Category=7-8b model, Backbone=Llama3.1-8B2025.11 | 47.5 | 43 | |
| DALKSetting=Lightweight Retriever, RT=72.62026.04 | 45.8 | 35.3 | |
| GRAGSetting=Lightweight Retriever, RT=7.22026.04 | 45.6 | 42.2 | |
| Interactive-KBQAModel Scale Category=13b model, Backbone=LLaMA2-13b2025.11 | 45.6 | 42.5 | |
| Interactive-KBQAModel Scale Category=7-8b model, Backbone=Mistral-7B2025.11 | 44 | 39.9 | |
| G-RetrieverSetting=Lightweight Retriever, RT=6.72026.04 | 42.1 | 40.8 | |
| Llama3.3-70BSetting=LLM without KG, RT=3.62026.04 | 39.5 | 34.1 | |
| ChatGPT+Few-shotModel Category=Closed-source business model2025.11 | 38.5 | 28 | |
| ChatGPTModel Category=Closed-source business model2025.11 | 34.7 | 30.2 | |
| Qwen2.5-72BSetting=LLM without KG, RT=4.32026.04 | 32.4 | 27.3 | |
| LLaMA2-7bModel Scale Category=7-8b model2025.11 | 28.4 | 21.4 | |
| LLaMA3.1-8bModel Scale Category=7-8b model2025.11 | 28.1 | 22.4 | |
| Qwen2-7bModel Scale Category=7-8b model2025.11 | 25.3 | 21.6 | |
| GSRModel Category=Hybrid System, Backbone=T5-3b + LLaMA2-7b2025.11 | — | 27.6 |