Multi-Hop Knowledge Graph Question Answering on GrailQA
94.4Hits@1PoG
Evaluation Results
| Method | Links | |
|---|---|---|
| PoGClass=ICL, LLM=GPT-4, External Knowledge=With external knowledge2024.10 | 94.4 | |
| PoGClass=ICL, LLM=GPT-3.5-Turbo, External Knowledge=With external knowledge2024.10 | 91.6 | |
| PoG-EClass=ICL, LLM=GPT-4, External Knowledge=With external knowledge2024.10 | 91.4 | |
| PoG-EClass=ICL, LLM=GPT-3.5-Turbo, External Knowledge=With external knowledge2024.10 | 87.6 | |
| PrivGemoType=Proposed, Brain LLM=DeepSeek-V3, Hand LLM=Qwen3-32b2026.01 | 86.1 | |
| PrivGemoType=Proposed, Brain LLM=GPT-3.5-Turbo, Hand LLM=Qwen3-32b2026.01 | 84.5 | |
| PrivGemoType=Proposed, Brain LLM=GPT-4o-mini, Hand LLM=Qwen3-32b2026.01 | 83.6 | |
| Logits-to-LogicBackbone LLM=LLaMA3.1-8b2025.11 | 82 | |
| ToG/ToG-RClass=ICL, LLM=GPT-4, External Knowledge=With external knowledge2024.10 | 81.4 | |
| ToGType=KG-centric RAG, LLM=GPT-42026.01 | 81.4 | |
| ToGMethod Paradigm=Agentic Reasoning, Backbone LLM=GPT42025.11 | 81.4 | |
| ToG-RMethod Paradigm=Agentic Reasoning, Backbone LLM=GPT42025.11 | 80.3 | |
| DoGMethod Paradigm=Agentic Reasoning2025.11 | 80 | |
| ARoGType=Privacy-aware KGQA, LLM=GPT-4o-mini2026.01 | 78.7 | |
| DARAMethod Paradigm=Agentic Reasoning, Backbone LLM=LLaMA2-13b2025.11 | 77 | |
| PoGType=KG-centric RAG, LLM=GPT-3.5-Turbo2026.01 | 76.5 | |
| PoGMethod Paradigm=Agentic Reasoning, Backbone LLM=ChatGPT2025.11 | 76.5 | |
| Prior FT SOTAClass=SL, External Knowledge=With external knowledge2024.10 | 75.4 | |
| ToG/ToG-RClass=ICL, LLM=GPT-3.5-Turbo, External Knowledge=With external knowledge2024.10 | 68.7 | |
| ToGType=KG-centric RAG, LLM=GPT-3.5-Turbo2026.01 | 68.7 | |
| ToGMethod Paradigm=Agentic Reasoning, Backbone LLM=ChatGPT2025.11 | 68.7 | |
| KB-BINDERClass=ICL, LLM=Codex, External Knowledge=With external knowledge2024.10 | 58.5 | |
| ToG-RMethod Paradigm=Agentic Reasoning, Backbone LLM=ChatGPT2025.11 | 56.4 | |
| o3-mini2025.05 | 43.8 | |
| Qwen2.5-72B2025.05 | 36.1 | |
| QwQ-32B2025.05 | 35.8 | |
| Qwen2.5-32B + fs1Protocol=fs12025.05 | 34.4 | |
| Qwen2.5-32B + rtProtocol=rt2025.05 | 34.2 | |
| R1-70B2025.05 | 34 | |
| Qwen2.5-32B2025.05 | 33.4 | |
| Qwen2.5-32B + cotProtocol=CoT2025.05 | 33.2 | |
| Qwen2.5-14B + rtProtocol=rt2025.05 | 33.1 | |
| Qwen2.5-14B + fs1Protocol=fs12025.05 | 32.8 | |
| Qwen2.5-14B + cotProtocol=CoT2025.05 | 32.2 | |
| Qwen2.5-14B2025.05 | 31.8 | |
| Qwen2.5-7B + fs1Protocol=fs12025.05 | 30.3 | |
| Qwen2.5-7B + rtProtocol=rt2025.05 | 30 | |
| Qwen2.5-7B2025.05 | 29.9 | |
| SCLLM=GPT-3.5-Turbo, External Knowledge=Without external knowledge2024.10 | 29.6 | |
| SCType=LLM-only, LLM=GPT-3.5-Turbo2026.01 | 29.6 | |
| SCMethod Paradigm=LLMs Reasoning, Backbone LLM=ChatGPT2025.11 | 29.6 | |
| Qwen2.5-7B + cotProtocol=CoT2025.05 | 29.5 | |
| IO promptLLM=GPT-3.5-Turbo, External Knowledge=Without external knowledge2024.10 | 29.4 | |
| IO promptType=LLM-only, LLM=GPT-3.5-Turbo2026.01 | 29.4 | |
| IO promptMethod Paradigm=LLMs Reasoning, Backbone LLM=ChatGPT2025.11 | 29.4 | |
| CoTLLM=GPT-3.5-Turbo, External Knowledge=Without external knowledge2024.10 | 28.1 | |
| CoTType=LLM-only, LLM=GPT-3.5-Turbo2026.01 | 28.1 | |
| CoTMethod Paradigm=LLMs Reasoning, Backbone LLM=ChatGPT2025.11 | 28.1 | |
| Qwen2.5-3B + rtProtocol=rt2025.05 | 27.9 | |
| Qwen2.5-3B + fs1Protocol=fs12025.05 | 25.3 | |
| Qwen2.5-3B2025.05 | 25.2 | |
| Qwen2.5-3B + cotProtocol=CoT2025.05 | 24.8 | |
| SmolLM2-1.7B + rtProtocol=rt2025.05 | 22.3 | |
| SmolLM2-1.7B2025.05 | 21.9 | |
| SmolLM2-1.7B + fs1Protocol=fs12025.05 | 21.8 | |
| Qwen2.5-1.5B + cotProtocol=CoT2025.05 | 21.6 | |
| Qwen2.5-1.5B + rtProtocol=rt2025.05 | 21.2 | |
| SmolLM2-1.7B + cotProtocol=CoT2025.05 | 20.9 | |
| Qwen2.5-1.5B2025.05 | 20.8 | |
| Qwen2.5-1.5B + fs1Protocol=fs12025.05 | 20.4 | |
| SmolLM2-360M + cotProtocol=CoT2025.05 | 16.9 | |
| Qwen2.5-0.5B + fs1Protocol=fs12025.05 | 16.6 | |
| SmolLM2-360M2025.05 | 16.4 | |
| SmolLM2-360M + rtProtocol=rt2025.05 | 15.6 | |
| Qwen2.5-0.5B + rtProtocol=rt2025.05 | 15.5 | |
| Qwen2.5-0.5B + cotProtocol=CoT2025.05 | 14.1 | |
| SmolLM2-360M + fs1Protocol=fs12025.05 | 13.8 | |
| Qwen2.5-0.5B2025.05 | 12.7 |