Knowledge Graph Question Answering on SimpleQA Freebase-based (test)
84.8Hits@1SoG
Evaluation Results
| Method | Links | |
|---|---|---|
| SoGBackbone=GPT-4o2025.10 | 84.8 | |
| EffiQABackbone=GPT-42025.10 | 76.5 | |
| ReKnoSBackbone=GPT-42025.10 | 69.3 | |
| KG-R1Backbone Model=Qwen2.5 3B, Training Dataset=WebQSP, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 69.18 | |
| ReknoSBackbone Model=Qwen 2.5-72B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 68.4 | |
| ReknoSBackbone Model=Qwen 3-235B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 68.01 | |
| PoGBackbone Model=Qwen 3-235B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 67.9 | |
| Think-on-GraphBackbone=GPT-42025.10 | 66.7 | |
| PoGBackbone Model=Qwen 2.5-7B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 66.38 | |
| PoGBackbone Model=Qwen 2.5-72B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 65.67 | |
| ToGBackbone Model=Qwen 2.5-72B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 57.83 | |
| KG-R1Backbone Model=Qwen2.5 3B, Training Dataset=CWQ, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 57.31 | |
| Think-on-GraphBackbone=GPT-4o, Reproduced status=reproduced2025.10 | 57.2 | |
| ReknoSBackbone Model=Qwen 2.5-7B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 56.43 | |
| ToGBackbone Model=Qwen 2.5-7B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 52.37 | |
| ToGBackbone Model=Qwen 3-235B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 41.27 | |
| Self-ConsistencyBackbone=GPT-4o2025.10 | 35.1 | |
| IO PromptBackbone=GPT-4o2025.10 | 34.3 | |
| Chain-of-ThoughtBackbone=GPT-4o2025.10 | 32.6 | |
| RoGBackbone Model=Qwen 2.5-72B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 20.4 | |
| RoGBackbone Model=Qwen 3-235B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 18.93 | |
| SCBackbone Model=Qwen 3-235B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 14.73 | |
| CoTBackbone Model=Qwen 3-235B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 14.4 | |
| IOBackbone Model=Qwen 2.5-72B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 14.33 | |
| RoGBackbone Model=Qwen 2.5-7B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 12.97 | |
| SCBackbone Model=Qwen 2.5-72B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 11.6 | |
| IOBackbone Model=Qwen 3-235B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 11.37 | |
| CoTBackbone Model=Qwen 2.5-72B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 11.07 | |
| IOBackbone Model=Qwen 2.5-7B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 10.3 | |
| SCBackbone Model=Qwen 2.5-7B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 8.7 | |
| CoTBackbone Model=Qwen 2.5-7B, Zero-shot protocol=true, Maximum turn number=H=52025.09 | 7.07 |