Knowledge Base Question Answering on GraphQ (test)
69.47F1DARA (w/Mistral-7B)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DARA (w/Mistral-7B)Learning Mode=Fine-tuned LLM Agent, Backbone=Mistral-7B, Zero-shot=true, Entity Linker=Golden2024.06 | 69.47 | 56.96 | |
| DARA (w/Llama-2-13B)Learning Mode=Fine-tuned LLM Agent, Backbone=Llama-2-13B, Zero-shot=true, Entity Linker=Golden2024.06 | 67.34 | 55.57 | |
| Pangu (T5-base)Learning Mode=Bottom-up Parser, Data=full train data, Backbone=T5-base, Zero-shot=true, Entity Linker=Golden2024.06 | 66.7 | 56.06 | |
| AgentBench (GPT-4)Learning Mode=Off-the-shelf (in-context learning), Backbone=GPT-4, Zero-shot=true, Entity Linker=Golden2024.06 | 64.48 | 53.86 | |
| DARA (w/CodeLlama-7B)Learning Mode=Fine-tuned LLM Agent, Backbone=CodeLlama-7B, Zero-shot=true, Entity Linker=Golden2024.06 | 63.15 | 51.83 | |
| DARA (w/Llama-2-7B)Learning Mode=Fine-tuned LLM Agent, Backbone=Llama-2-7B, Zero-shot=true, Entity Linker=Golden2024.06 | 62.74 | 51.51 | |
| GAPDLLM=Llama-3.1-8B, Category=Fine-tune-based Methods2026.05 | 55.1 | — | |
| AgentLM-13BLearning Mode=Fine-tuned LLM Agent, Backbone=13B, Zero-shot=true, Entity Linker=Golden2024.06 | 54.35 | 44.34 | |
| KBQA-R1LLM=Llama-3.1-8B, Evaluation Protocol=Fine-tune-based2025.12 | 53.8 | — | |
| KBQA-R1LLM=Llama-3.1-8B, Category=Fine-tune-based Methods2026.05 | 53.8 | — | |
| AgentBench-7BLearning Mode=Fine-tuned LLM Agent, Backbone=7B, Zero-shot=true, Entity Linker=Golden2024.06 | 53.37 | 42.72 | |
| KBQA-R1 HarnessLLM=Kimi-K2.5, Category=Prompting Methods2026.05 | 53.3 | — | |
| KBQA-R1 HarnessLLM=Claude 4.6 Sonnet, Category=Prompting Methods2026.05 | 51.8 | — | |
| KBQA-R1 HarnessLLM=gpt-5.3-codex, Category=Prompting Methods2026.05 | 51.5 | — | |
| KBQA-o1LLM=Llama-3.1-8B, Evaluation Protocol=Fine-tune-based2025.12 | 48.7 | — | |
| KBQA-o1LLM=Llama-3.1-8B, Category=Fine-tune-based Methods2026.05 | 48.7 | — | |
| KBQA-R1 HarnessLLM=GLM-5, Category=Prompting Methods2026.05 | 48.5 | — | |
| ArcaneQALearning Mode=Bottom-up Parser, Data=full train data, Zero-shot=true, Entity Linker=Golden2024.06 | 47.5 | 37 | |
| CoTKRLLM=Llama-3-8B, Evaluation Protocol=Fine-tune-based2025.12 | 47.5 | — | |
| CoTKRLLM=Llama-3-8B, Category=Fine-tune-based Methods2026.05 | 47.5 | — | |
| AgentLM-7BLearning Mode=Fine-tuned LLM Agent, Backbone=7B, Zero-shot=true, Entity Linker=Golden2024.06 | 43.92 | 36.21 | |
| ArcaneQALLM=BERT-base, Evaluation Protocol=Fine-tune-based2025.12 | 31.8 | — | |
| ArcaneQALLM=BERT-base, Category=Fine-tune-based Methods2026.05 | 31.8 | — | |
| KB-CoderLLM=GPT-3.5-turbo, Evaluation Protocol=Prompting2025.12 | 31.1 | — | |
| KB-CoderLLM=GPT-3.5-turbo, Category=Prompting Methods2026.05 | 31.1 | — | |
| AgentBench (Llama-2-chat-70B)Learning Mode=Off-the-shelf (in-context learning), Backbone=Llama-2-chat-70B, Zero-shot=true, Entity Linker=Golden2024.06 | 30.33 | 25.63 | |
| KB-BINDERLLM=Codex-davinci-002, Evaluation Protocol=Prompting2025.12 | 27.1 | — | |
| KB-BINDERLLM=Codex-davinci-002, Category=Prompting Methods2026.05 | 27.1 | — | |
| BERT+RankingLLM=BERT-base, Evaluation Protocol=Fine-tune-based2025.12 | 25 | — | |
| BERT+RankingLLM=BERT-base, Category=Fine-tune-based Methods2026.05 | 25 | — | |
| SPARQALLM=BERT-base, Evaluation Protocol=Fine-tune-based2025.12 | 21.5 | — | |
| SPARQALLM=BERT-base, Category=Fine-tune-based Methods2026.05 | 21.5 | — | |
| Pangu (T5-Large)Learning Mode=Bottom-up Parser, Data=full train data, Backbone=T5-Large, Zero-shot=true, Entity Linker=Golden2024.06 | — | 55.57 |