Knowledge Base Question Answering on GrailQA Freebase (test)
86.9Hits@1SoG
Evaluation Results
| Method | Links | |
|---|---|---|
| SoGBackbone=GPT-4o2025.10 | 86.9 | |
| Plan-on-GraphBackbone=GPT-42025.10 | 84.7 | |
| ReKnoSBackbone=GPT-42025.10 | 82.7 | |
| Think-on-GraphBackbone=GPT-42025.10 | 81.4 | |
| Debate-on-GraphBackbone=GPT-42025.10 | 80 | |
| EffiQABackbone=GPT-42025.10 | 78.4 | |
| PoGBackbone Model=Qwen 3-235B2025.09 | 77.93 | |
| ReknoSBackbone Model=Qwen 3-235B2025.09 | 75.67 | |
| PoGBackbone Model=Qwen 2.5-72B2025.09 | 75.3 | |
| ReknoSBackbone Model=Qwen 2.5-72B2025.09 | 73.67 | |
| iQUESTBackbone=GPT-4o2025.10 | 73.5 | |
| ToGBackbone Model=Qwen 2.5-72B2025.09 | 71.53 | |
| PoGBackbone Model=Qwen 2.5-7B2025.09 | 69.88 | |
| ReknoSBackbone Model=Qwen 2.5-7B2025.09 | 69.58 | |
| KG-R1Backbone Model=Qwen2.5 3B, Training Dataset=WebQSP2025.09 | 68.55 | |
| Think-on-GraphBackbone=GPT-4o, Reproduced status=reproduced2025.10 | 65.5 | |
| KG-R1Backbone Model=Qwen2.5 3B, Training Dataset=CWQ2025.09 | 64.84 | |
| GPT-4o-mini + KGStrategy=KG-Augmented without Fine-Tuning2026.03 | 60 | |
| DeepSeek-R1-Distill-Llama-70B + KGSize=70B, Strategy=KG-Augmented without Fine-Tuning2026.03 | 59.02 | |
| LLaMA-3.3-70B + KGSize=70B, Strategy=KG-Augmented without Fine-Tuning2026.03 | 57.7 | |
| ToGBackbone Model=Qwen 2.5-7B2025.09 | 54.73 | |
| ToGBackbone Model=Qwen 3-235B2025.09 | 53.2 | |
| KG-Hopper w/Qwen-2.5-7BSize=7B, Strategy=KG-Augmented with RL Fine-Tuning2026.03 | 50.1 | |
| LLaMA-3.1-8B (SFT) + KGSize=8B, Strategy=KG-Augmented with Supervised Fine-Tuning2026.03 | 47.7 | |
| Qwen-2.5-7B (SFT) + KGSize=7B, Strategy=KG-Augmented with Supervised Fine-Tuning2026.03 | 46.18 | |
| Qwen-2.5-7B + KGSize=7B, Strategy=KG-Augmented without Fine-Tuning2026.03 | 41.1 | |
| LLaMA-3.1-8B + KGSize=8B, Strategy=KG-Augmented without Fine-Tuning2026.03 | 40.4 | |
| Chain-of-ThoughtBackbone=GPT-4o2025.10 | 38 | |
| Self-ConsistencyBackbone=GPT-4o2025.10 | 37.4 | |
| GPT-4o-miniStrategy=LLM Prompting Only2026.03 | 36.22 | |
| IO PromptBackbone=GPT-4o2025.10 | 35.3 | |
| GPT-4oStrategy=LLM Prompting Only2026.03 | 35.01 | |
| LLaMA-3.3-70BSize=70B, Strategy=LLM Prompting Only2026.03 | 33.79 | |
| DeepSeek-R1-Distill-Llama-70BSize=70B, Strategy=LLM Prompting Only2026.03 | 31.7 | |
| Qwen-2.5-7BSize=7B, Strategy=LLM Prompting Only2026.03 | 29.53 | |
| LLaMA-3.1-8BSize=8B, Strategy=LLM Prompting Only2026.03 | 28.35 | |
| RoGBackbone Model=Qwen 2.5-72B2025.09 | 22.63 | |
| RoGBackbone Model=Qwen 3-235B2025.09 | 18.1 | |
| RoGBackbone Model=Qwen 2.5-7B2025.09 | 17.67 | |
| SCBackbone Model=Qwen 3-235B2025.09 | 15.73 | |
| CoTBackbone Model=Qwen 3-235B2025.09 | 14.6 | |
| IOBackbone Model=Qwen 3-235B2025.09 | 12.8 | |
| IOBackbone Model=Qwen 2.5-72B2025.09 | 11.63 | |
| SCBackbone Model=Qwen 2.5-72B2025.09 | 11.43 | |
| CoTBackbone Model=Qwen 2.5-72B2025.09 | 10.53 | |
| SCBackbone Model=Qwen 2.5-7B2025.09 | 8.6 | |
| IOBackbone Model=Qwen 2.5-7B2025.09 | 8.07 | |
| CoTBackbone Model=Qwen 2.5-7B2025.09 | 7.73 |