Knowledge Graph Question Answering on WebQSP (test)
93.2HitAURA
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AURAFramework=LlamaIndex2026.01 | 93.2 | — | — | — | 100 | — | — | — | — | — | — | — | |
| AURAFramework=GraphRAG-M2026.01 | 92.8 | — | — | — | 100 | — | — | — | — | — | — | — | |
| TRACEType=LLMs with KGs2026.04 | 91.6 | — | 81.7 | — | — | — | — | — | — | — | — | — | |
| RL-DoublyCalKG Evidence=Constrained Relational Path, UQ Method=Vanilla2026.01 | 91.5 | 84.8 | 76.7 | 4.5 | — | — | — | — | — | — | — | — | |
| RL-DoublyCalKG Evidence=Constrained Relational Path, UQ Method=CoT2026.01 | 91.5 | 85 | 76.8 | 3.9 | — | — | — | — | — | — | — | — | |
| STEMEvaluation Protocol=Our Proposed Method, Backbone LLM=GPT-4o2026.04 | 90.94 | — | 76.18 | — | — | — | — | — | — | — | — | — | |
| TopoRAG w/ LoRASetting=Tuned LLM2026.02 | 90.66 | — | — | — | — | — | — | — | — | — | — | — | |
| ProgRAGEvaluation Protocol=Prompting, Backbone LLM=gpt-4o-mini2026.04 | 90.4 | — | — | — | — | — | — | — | — | — | — | — | |
| SFT-DoublyCalKG Evidence=Constrained Relational Path, UQ Method=CoT2026.01 | 90.1 | 81.3 | 72.1 | 3.5 | — | — | — | — | — | — | — | — | |
| SFT-DoublyCalKG Evidence=Constrained Relational Path, UQ Method=Vanilla2026.01 | 90 | 81 | 72.6 | 3.1 | — | — | — | — | — | — | — | — | |
| RoGKG Evidence=Relational Path, UQ Method=CoT2026.01 | 89.9 | 78.5 | 68.2 | 18.9 | — | — | — | — | — | — | — | — | |
| RL-DoublyCalKG Evidence=Constrained Relational Path, UQ Method=Self-Probing2026.01 | 89.9 | 83 | 79.3 | 6.8 | — | — | — | — | — | — | — | — | |
| SubgraphRAGKG Evidence=Subgraph, UQ Method=Self-Probing2026.01 | 89.6 | 80.7 | 74.9 | 12.3 | — | — | — | — | — | — | — | — | |
| RoGKG Evidence=Relational Path, UQ Method=Vanilla2026.01 | 89.3 | 77.6 | 67.1 | 19.6 | — | — | — | — | — | — | — | — | |
| SubgraphRAGKG Evidence=Subgraph, UQ Method=CoT2026.01 | 89 | 81 | 77.1 | 10.6 | — | — | — | — | — | — | — | — | |
| SubgraphRAGKG Evidence=Subgraph, UQ Method=Vanilla2026.01 | 88.8 | 81.3 | 77.3 | 11.1 | — | — | — | — | — | — | — | — | |
| SFT-DoublyCalKG Evidence=Constrained Relational Path, UQ Method=Self-Probing2026.01 | 88.5 | 79.4 | 76.6 | 7.9 | — | — | — | — | — | — | — | — | |
| RoGEvaluation Protocol=Finetuning, Backbone LLM=GPT-4o2026.04 | 88.09 | — | 70.12 | — | — | — | — | — | — | — | — | — | |
| STEMEvaluation Protocol=Our Proposed Method, Backbone LLM=Llama-3.1-70B2026.04 | 88.08 | — | 74.62 | — | — | — | — | — | — | — | — | — | |
| RoGKG Evidence=Relational Path, UQ Method=Self-Probing2026.01 | 87.5 | 76.6 | 73.5 | 13.9 | — | — | — | — | — | — | — | — | |
| DPType=LLMs with KGs2026.04 | 87.5 | — | 81.4 | — | — | — | — | — | — | — | — | — | |
| TopoRAGSetting=Frozen LLM w/ PT2026.02 | 87.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RwTType=LLMs with KGs2026.04 | 87 | — | 79.7 | — | — | — | — | — | — | — | — | — | |
| STEMEvaluation Protocol=Our Proposed Method, Backbone LLM=Llama-3.1-8B2026.04 | 86.63 | — | 71.05 | — | — | — | — | — | — | — | — | — | |
| SubgraphRAGSetting=Frozen LLM w/ PT2026.02 | 86.61 | — | — | — | — | — | — | — | — | — | — | — | |
| GNN-RAGEvaluation Protocol=Finetuning, Backbone LLM=Llama2-7B2026.04 | 86.4 | — | 69 | — | — | — | — | — | — | — | — | — | |
| RoGEvaluation Protocol=Finetuning, Backbone LLM=Llama-3.1-70B2026.04 | 86.1 | — | 68.87 | — | — | — | — | — | — | — | — | — | |
| GNN-RAGSetting=Tuned LLM2026.02 | 85.7 | — | — | — | — | — | — | — | — | — | — | — | |
| FiDeLiSEvaluation Protocol=Prompting, Backbone LLM=gpt-4-turbo2026.04 | 84.39 | — | 78.32 | — | — | — | — | — | — | — | — | — | |
| FiDeLisType=LLMs with KGs2026.04 | 84.3 | — | 78.3 | — | — | — | — | — | — | — | — | — | |
| LightProfEvaluation Protocol=Finetuning, Backbone LLM=Llama3-8B2026.04 | 83.8 | — | — | — | — | — | — | — | — | — | — | — | |
| RoGEvaluation Protocol=Finetuning, Backbone LLM=Llama2-Chat-7B2026.04 | 83.15 | — | 69.81 | — | — | — | — | — | — | — | — | — | |
| SubgraphRAGEvaluation Protocol=Prompting, Backbone LLM=gpt-4o2026.04 | 83.1 | — | — | — | — | — | — | — | — | — | — | — | |
| GNN-RAGType=LLMs with KGs2026.04 | 82.8 | — | 73.5 | — | — | — | — | — | — | — | — | — | |
| DECAFType=Semantic Parsing2026.04 | 82.1 | — | 78.8 | — | — | — | — | — | — | — | — | — | |
| DeCAFFEvaluation Protocol=Finetuning, Backbone LLM=FiD-3B2026.04 | 82.1 | — | — | — | — | — | — | — | — | — | — | — | |
| PoGEvaluation Protocol=Prompting, Backbone LLM=gpt-3.52026.04 | 82 | — | — | — | — | — | — | — | — | — | — | — | |
| ToGType=LLMs with KGs2026.04 | 81.9 | — | 76 | — | — | — | — | — | — | — | — | — | |
| DualRType=LLMs with KGs2026.04 | 81.5 | — | 71.6 | — | — | — | — | — | — | — | — | — | |
| ReKnoSEvaluation Protocol=Prompting, Backbone LLM=gpt-3.52026.04 | 81.1 | — | — | — | — | — | — | — | — | — | — | — | |
| RoGType=LLMs with KGs2026.04 | 80.8 | — | 70.8 | — | — | — | — | — | — | — | — | — | |
| MFCEvaluation Protocol=Prompting, Backbone LLM=gpt-4o-mini2026.04 | 78.9 | — | — | — | — | — | — | — | — | — | — | — | |
| ReasoningLMType=LLMs with KGs2026.04 | 78.5 | — | 71 | — | — | — | — | — | — | — | — | — | |
| NutreaType=LLMs with KGs2026.04 | 77.4 | — | 72.7 | — | — | — | — | — | — | — | — | — | |
| UniKGQAType=LLMs with KGs2026.04 | 77.2 | — | 72.2 | — | — | — | — | — | — | — | — | — | |
| ChatGPT+CoTType=LLMs2026.04 | 75.6 | — | — | — | — | — | — | — | — | — | — | — | |
| LLM ReasonerKG Evidence=No Augmentation, UQ Method=CoT2026.01 | 75.4 | 53.9 | 44.4 | 26.6 | — | — | — | — | — | — | — | — | |
| ToGEvaluation Protocol=Prompting, Backbone LLM=gpt-3.5-turbo2026.04 | 75.08 | — | 72.32 | — | — | — | — | — | — | — | — | — | |
| LLM ReasonerKG Evidence=No Augmentation, UQ Method=Vanilla2026.01 | 74.7 | 53.1 | 44.6 | 27.7 | — | — | — | — | — | — | — | — | |
| NSMEvaluation Protocol=Finetuning2026.04 | 74.31 | — | — | — | — | — | — | — | — | — | — | — | |
| GPT-4o+CoTEvaluation Protocol=CoT, Backbone LLM=GPT-4o2026.04 | 74.12 | — | 64.25 | — | — | — | — | — | — | — | — | — | |
| LLM ReasonerKG Evidence=No Augmentation, UQ Method=Self-Probing2026.01 | 74.1 | 52.8 | 50.2 | 36.5 | — | — | — | — | — | — | — | — | |
| G-Retriever w/ LoRASetting=Tuned LLM2026.02 | 73.79 | — | — | — | — | — | — | — | — | — | — | — | |
| KD-CoTEvaluation Protocol=Finetuning, Backbone LLM=T5-large2026.04 | 73.7 | — | 50.2 | — | — | — | — | — | — | — | — | — | |
| QGGType=Semantic Parsing2026.04 | 73 | — | 73.8 | — | — | — | — | — | — | — | — | — | |
| GRAGEvaluation Protocol=Finetuning, Backbone LLM=LLaMA2-7B2026.04 | 72.75 | — | 50.41 | — | — | — | — | — | — | — | — | — | |
| KapingEvaluation Protocol=Prompting, Backbone LLM=gpt-3.5-turbo2026.04 | 72.42 | — | 65.12 | — | — | — | — | — | — | — | — | — | |
| KAPINGType=LLMs with KGs2026.04 | 72.4 | — | 65.1 | — | — | — | — | — | — | — | — | — | |
| GPT-4o+FewshotEvaluation Protocol=Fewshot, Backbone LLM=GPT-4o2026.04 | 71.68 | — | 63.7 | — | — | — | — | — | — | — | — | — | |
| TransferNetType=Semantic Parsing2026.04 | 71.4 | — | — | — | — | — | — | — | — | — | — | — | |
| G-RetrieverSetting=Frozen LLM w/ PT2026.02 | 70.49 | — | — | — | — | — | — | — | — | — | — | — | |
| G-RetEvaluation Protocol=Prompting, Backbone LLM=Llama2-7B2026.04 | 70.16 | — | 50.23 | — | — | — | — | — | — | — | — | — | |
| SR+NSM+E2EType=Retrieval2026.04 | 69.5 | — | 64.1 | — | — | — | — | — | — | — | — | — | |
| SR+NSMType=Retrieval2026.04 | 68.9 | — | 64.1 | — | — | — | — | — | — | — | — | — | |
| NSMType=Semantic Parsing2026.04 | 68.7 | — | 62.8 | — | — | — | — | — | — | — | — | — | |
| KD-CoTType=LLMs with KGs2026.04 | 68.6 | — | 52.5 | — | — | — | — | — | — | — | — | — | |
| PullNetType=Retrieval2026.04 | 68.1 | — | — | — | — | — | — | — | — | — | — | — | |
| ChatGPTType=LLMs2026.04 | 66.8 | — | — | — | — | — | — | — | — | — | — | — | |
| EmbedKGQAType=Semantic Parsing2026.04 | 66.6 | — | — | — | — | — | — | — | — | — | — | — | |
| GraftNetType=Retrieval2026.04 | 66.4 | — | 60.4 | — | — | — | — | — | — | — | — | — | |
| LoRASetting=Tuned LLM2026.02 | 66.03 | — | — | — | — | — | — | — | — | — | — | — | |
| DoGType=LLMs with KGs2026.04 | 65.4 | — | 55.6 | — | — | — | — | — | — | — | — | — | |
| GPT-4oEvaluation Protocol=Zero-shot, Backbone LLM=GPT-4o2026.04 | 61.8 | — | 43.6 | — | — | — | — | — | — | — | — | — | |
| GraphTokenSetting=Frozen LLM w/ PT2026.02 | 57.05 | — | — | — | — | — | — | — | — | — | — | — | |
| KGT5Type=Semantic Parsing2026.04 | 56.1 | — | — | — | — | — | — | — | — | — | — | — | |
| KAPINGSetting=Inference-only2026.02 | 52.64 | — | — | — | — | — | — | — | — | — | — | — | |
| Alpaca-7BType=LLMs2026.04 | 51.8 | — | — | — | — | — | — | — | — | — | — | — | |
| Zero-CoTSetting=Inference-only2026.02 | 51.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Prompt tuningSetting=Frozen LLM w/ PT2026.02 | 48.34 | — | — | — | — | — | — | — | — | — | — | — | |
| KV-MemType=Semantic Parsing2026.04 | 46.7 | — | 34.5 | — | — | — | — | — | — | — | — | — | |
| Zero-shotSetting=Inference-only2026.02 | 41.06 | — | — | — | — | — | — | — | — | — | — | — | |
| CoT-BAGSetting=Inference-only2026.02 | 39.6 | — | — | — | — | — | — | — | — | — | — | — | |
| Flan-T5-xlType=LLMs2026.04 | 31 | — | — | — | — | — | — | — | — | — | — | — | |
| Llama3-8BType=LLMs2026.04 | 30.3 | — | 25.7 | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-7BType=LLMs2026.04 | 28.4 | — | 23.7 | — | — | — | — | — | — | — | — | — | |
| ChatGPTType=LLMs2026.06 | — | — | 43.5 | — | — | — | 59.3 | — | — | — | — | — | |
| ConceptFormerTraining=QA fine-tuned on WebQSP2026.05 | — | — | — | — | — | — | 7.6 | — | 28.3 | — | — | — | |
| CoTBase Model=ChatGPT2026.06 | — | — | — | — | — | — | 62.2 | — | — | — | — | 57.7 | |
| DeepSeek-v3Type=LLMs2026.06 | — | — | 43.9 | — | — | — | 64 | — | — | — | — | — | |
| FiDeLiSType=Interactive2026.04 | — | — | — | — | — | 34.97 | — | — | — | — | — | — | |
| GCRType=KGs+LLMs, Backbone=Llama-2-7B + GPT-4o2026.06 | — | — | 71.5 | — | — | — | 87.5 | — | — | — | — | — | |
| GCR-BRSetting=Retrieval-only2026.06 | — | — | 58.03 | — | — | — | 92.19 | — | — | — | — | — | |
| GNN-RAGType=Retrieval2026.04 | — | — | — | — | — | 2.64 | — | — | — | — | — | — | |
| GNN-RAGType=KGs+LLMs, Backbone=Llama-2-7B2026.06 | — | — | 71.3 | — | — | — | 85.7 | — | — | — | — | — | |
| GPT-4oType=LLMs2026.06 | — | — | 43.6 | — | — | — | 61.8 | — | — | — | — | — | |
| GraftNetType=Retrieval-based2026.06 | — | — | 62.4 | — | — | — | 66.7 | — | — | — | — | — | |
| IO PromptBase Model=ChatGPT2026.06 | — | — | — | — | — | — | 63.3 | — | — | — | — | 58.2 | |
| KGT5Type=Embedding-based2026.06 | — | — | — | — | — | — | 56.1 | — | — | — | — | — | |
| KoRe-baseConfiguration=KoRe, Backbone=Qwen3-8B2026.05 | — | — | — | — | — | — | 23.8 | 40.7 | 49.1 | 58.5 | 62.9 | — | |
| KoRe-QAConfiguration=KoRe, Backbone=Qwen3-8B2026.05 | — | — | — | — | — | — | 51.4 | 73.2 | 79.3 | 86 | 62.9 | — |