Knowledge Base Question Answering on WebQSP (test)
88.6Hit@1FRAG
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FRAGBackbone=Llama-3-70B-Instruct, Type=Modular KG-RAG2025.01 | 88.6 | — | — | — | — | — | |
| FRAG-FBackbone=Llama-3-70B-Instruct, Type=Modular KG-RAG2025.01 | 88.6 | — | — | — | — | — | |
| FRAG-FBackbone=Llama-3-8B-Instruct, Type=Modular KG-RAG2025.01 | 87.8 | — | — | — | — | — | |
| FRAGBackbone=Llama-3-8B-Instruct, Type=Modular KG-RAG2025.01 | 87.7 | — | — | — | — | — | |
| PoGMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-42024.10 | 87.3 | — | — | — | — | — | |
| EMBRAG2026.02 | 86.81 | 67.29 | — | 76.96 | 69.2 | 76.52 | |
| FRAGBackbone=GPT-4o-mini, Type=Modular KG-RAG2025.01 | 86.7 | — | — | — | — | — | |
| FRAG-FBackbone=GPT-4o-mini, Type=Modular KG-RAG2025.01 | 86.7 | — | — | — | — | — | |
| ChatKBQABackbone=Llama-2-7B, Fine-tuning=LoRA, Beam size=15, Oracle Entity Linking=true2023.10 | 86.4 | 83.5 | — | 77.8 | — | — | |
| RoG2026.02 | 86.29 | 70.48 | — | 76.48 | 74.24 | 76.71 | |
| RoGMethod Category=Fine-Tuned KG-Augmented LLM2024.10 | 85.7 | — | — | — | — | — | |
| RoGType=Coupled KG-RAG2025.01 | 85.7 | — | — | — | — | — | |
| ChatKBQABackbone=Llama-2-7B, Fine-tuning=LoRA, Beam size=15, Oracle Entity Linking=false2023.10 | 83.2 | 79.8 | — | 73.8 | — | — | |
| ToGMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-42024.10 | 82.6 | — | — | — | — | — | |
| ToGMethod Category=LLM-based, Oracle Entity Linking=true2023.10 | 82.6 | — | — | — | — | — | |
| FRAGBackbone=ChatGPT, Type=Modular KG-RAG2025.01 | 82.3 | — | — | — | — | — | |
| FRAG-FBackbone=ChatGPT, Type=Modular KG-RAG2025.01 | 82.3 | — | — | — | — | — | |
| DeCAFMethod Category=Fine-Tuned KG-Augmented LLM2024.10 | 82.1 | — | — | — | — | — | |
| DecAFMethod Category=SP-based (seq2seq), Oracle Entity Linking=false2023.10 | 82.1 | 78.8 | — | — | — | — | |
| PoGMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-3.5 or others2024.10 | 82 | — | — | — | — | — | |
| FRAG-FBackbone=Llama-2-70b-chat-hf, Type=Modular KG-RAG2025.01 | 81.3 | — | — | — | — | — | |
| FRAGBackbone=Llama-2-70b-chat-hf, Type=Modular KG-RAG2025.01 | 81.2 | — | — | — | — | — | |
| DECAFApproach Type=multi-stage, LLM, Gold Entity Mentions=false2024.06 | 80.7 | 77.1 | — | — | — | — | |
| GNN-RAGType=Coupled KG-RAG2025.01 | 80.6 | — | — | — | — | — | |
| SKP2023.08 | 79.6 | — | — | — | — | — | |
| SKPIAM=excluded2023.08 | 79.3 | — | — | — | — | — | |
| UniK-QAbackbone=T5-large, knowledge source=Freebase2020.12 | 79.1 | — | — | — | — | — | |
| SKPKCD=excluded2023.08 | 79.1 | — | — | — | — | — | |
| UniKGQAMethod Category=Fine-Tuned KG-Augmented LLM2024.10 | 79.1 | — | — | — | — | — | |
| SKPKM=excluded2023.08 | 78.7 | — | — | — | — | — | |
| SKPKM=excluded, KCD=excluded2023.08 | 78.4 | — | — | — | — | — | |
| UniK-QAReproduced=true2023.08 | 77.9 | — | — | — | — | — | |
| SKPKM=excluded, KCD=excluded, IAM=excluded2023.08 | 77.9 | — | — | — | — | — | |
| UniKGQAMethod Category=SP-based (step-wise), Oracle Entity Linking=true2023.10 | 77.2 | 72.2 | — | — | — | — | |
| UniKGQAType=Traditional KGQA2025.01 | 77.2 | — | — | — | — | — | |
| HGNetBackbone=BERT-Base2021.11 | 76.9 | 76.6 | — | — | — | — | |
| UniK-QAbackbone=T5-base, knowledge source=Freebase2020.12 | 76.7 | — | — | — | — | — | |
| FRAG-FBackbone=Llama-2-7b-chat-hf, Type=Modular KG-RAG2025.01 | 76.7 | — | — | — | — | — | |
| FRAGBackbone=Llama-2-7b-chat-hf, Type=Modular KG-RAG2025.01 | 76.6 | — | — | — | — | — | |
| ToGMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-3.5 or others2024.10 | 76.2 | — | — | — | — | — | |
| ToGBackbone=ChatGPT, Type=Modular KG-RAG2025.01 | 76.2 | — | — | — | — | — | |
| EmQLknowledge source=Freebase, uses oracle entities=true2020.12 | 75.5 | — | — | — | — | — | |
| EMQL2023.08 | 75.5 | — | — | — | — | — | |
| TIARAMethod Category=Fine-Tuned KG-Augmented LLM2024.10 | 75.2 | — | — | — | — | — | |
| TIARAMethod Category=SP-based (seq2seq), Oracle Entity Linking=true2023.10 | 75.2 | 78.9 | — | — | — | — | |
| DECAFMode=Answer only2023.08 | 74.7 | — | — | — | — | — | |
| OursFull model=true, Backbone=BERT-base-cased2021.10 | 74.6 | 76.5 | — | — | — | — | |
| RE-KBQAVersion=Full pipeline2023.05 | 74.6 | 68.5 | — | — | — | — | |
| RE-KBQAMethod Category=Fine-Tuned KG-Augmented LLM2024.10 | 74.6 | — | — | — | — | — | |
| Program TransferMethod Category=SP-based (seq2seq), Oracle Entity Linking=true2023.10 | 74.6 | 76.5 | — | — | — | — | |
| Program TransferType=Traditional KGQA2025.01 | 74.6 | — | — | — | — | — | |
| KB-BINDERMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-3.5 or others2024.10 | 74.4 | — | — | — | — | — | |
| TeacherNet2021.10 | 74.3 | 67.4 | — | — | — | — | |
| NSM+h2021.11 | 74.3 | 67.4 | — | — | — | — | |
| BiNSM*Method Category=IR-Based2023.05 | 74.3 | 67.4 | — | — | — | — | |
| NSM2023.08 | 74.3 | — | — | — | — | — | |
| NSM+hMethod Category=IR-based, Oracle Entity Linking=true2023.10 | 74.3 | 67.4 | — | — | — | — | |
| FiDeLiSBackend Model=Mistral-7B, temperature=0.3, top_p=1.02024.05 | 74.11 | 70.23 | — | — | — | — | |
| QGG2021.10 | 74 | — | — | — | — | — | |
| KD-CoTMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-3.5 or others2024.10 | 73.7 | — | — | — | — | — | |
| KD-CoTBackbone=ChatGPT, Type=Modular KG-RAG2025.01 | 73.7 | — | — | — | — | — | |
| Vanilla LLMBackbone=Llama-3-70B-Instruct, Type=Modular KG-RAG2025.01 | 73.1 | — | — | — | — | — | |
| QGGMethod Category=SP-based (step-wise), Oracle Entity Linking=false2023.10 | 73 | 74 | — | — | — | — | |
| QGGType=Traditional KGQA2025.01 | 73 | — | — | — | — | — | |
| Bert-KBQA2023.08 | 72.9 | — | — | — | — | — | |
| NPI2021.10 | 72.6 | — | — | — | — | — | |
| StructGPTMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-3.5 or others2024.10 | 72.6 | — | — | — | — | — | |
| StructGPTMethod Category=LLM-based, Oracle Entity Linking=true2023.10 | 72.6 | — | — | — | — | — | |
| StructGPTApproach Type=multi-stage, LLM, Gold Entity Mentions=true2024.06 | 72.6 | — | — | — | — | — | |
| EmbededKGQA2023.08 | 72.5 | — | — | — | — | — | |
| InteractiveKBQAMethod Category=Prompting KG-Augmented LLM, Base LLM=GPT-42024.10 | 72.5 | — | — | — | — | — | |
| FiDeLiSBackend Model=Llama-2-13B, temperature=0.3, top_p=1.02024.05 | 72.34 | 69.78 | — | — | — | — | |
| TEXTRAY2021.10 | 72.2 | 60.3 | — | — | — | — | |
| CBR-SUBG2023.08 | 72.1 | — | — | — | — | — | |
| HGNetBackbone=GloVe + BiLSTM2021.11 | 71.7 | 71.7 | — | — | — | — | |
| STAGGMethod Category=SP-based (step-wise), Oracle Entity Linking=false2023.10 | 71.7 | — | — | — | — | — | |
| HGNetApproach Type=multi-stage, LLM, Gold Entity Mentions=true2024.06 | 71.7 | — | — | — | — | — | |
| ReTraCk*Method Category=IR-Based2023.05 | 71.6 | 71 | — | — | — | — | |
| TransferNetMethod Category=IR-based, Oracle Entity Linking=false2023.10 | 71.4 | — | — | — | — | — | |
| TransferNetType=Traditional KGQA2025.01 | 71.4 | — | — | — | — | — | |
| Ours-oOntology guidance=false, Backbone=BERT-base-cased2021.10 | 71.3 | 72 | — | — | — | — | |
| SPARKLEApproach Type=end-to-end, Gold Entity Mentions=false2024.06 | 71.2 | 71.1 | — | — | — | — | |
| HGNetType=Traditional KGQA2025.01 | 70.6 | — | — | — | — | — | |
| SPARKLEApproach Type=end-to-end, Gold Entity Mentions=false, Constraint Settings=w/o pruning2024.06 | 70.3 | 70.1 | — | — | — | — | |
| SR-KBQA*Method Category=IR-Based2023.05 | 69.5 | 64.1 | — | — | — | — | |
| GraftNet2023.08 | 69.5 | — | — | — | — | — | |
| SR-NSM2023.08 | 69.5 | — | — | — | — | — | |
| Subgraph RetrievalMethod Category=IR-based, Oracle Entity Linking=true2023.10 | 69.5 | 64.1 | — | — | — | — | |
| SR+NSM+E2EType=Traditional KGQA2025.01 | 69.5 | — | — | — | — | — | |
| Vanilla LLMBackbone=GPT-4o-mini, Type=Modular KG-RAG2025.01 | 69.2 | — | — | — | — | — | |
| NSM2021.10 | 69 | — | — | — | — | — | |
| Ours-paPre-training for argument parser=false, Backbone=BERT-base-cased2021.10 | 68.9 | 70.8 | — | — | — | — | |
| SR+NSMType=Traditional KGQA2025.01 | 68.9 | — | — | — | — | — | |
| ToGBackbone=Llama-2-70b-chat-hf, Type=Modular KG-RAG2025.01 | 68.9 | — | — | — | — | — | |
| KG-FlexApproach Type=end-to-end, Gold Entity Mentions=false2024.06 | 68.9 | — | — | — | — | — | |
| GraftNet2021.10 | 68.7 | 62.3 | — | — | — | — | |
| RE-KBQA_bVersion=Backbone network2023.05 | 68.7 | 62.8 | — | — | — | — | |
| NSMType=Traditional KGQA2025.01 | 68.7 | — | — | — | — | — | |
| GNN-KBQA*Method Category=SP-Based2023.05 | 68.5 | 68.9 | — | — | — | — | |
| NSMMethod Category=IR-Based2023.05 | 68.5 | 62.8 | — | — | — | — |