# Knowledge Base Question Answering on GrailQA i.i.d. (dev) Source: [Knowledge Base Question Answering on GrailQA i.i.d. (dev)]() Collection index: [llms.txt]() Data updated: 2026-05-29T09:14:34.959Z Evaluation of Knowledge Base Question Answering performance on the I.I.D. subset of the development split of the GrailQA dataset, reporting EM and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Knowledge Base Question Answering | | Dataset (including any stated split) | GrailQA i.i.d. (dev) | | Metric | F1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 14 of 14 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [F1](): Higher is better; unit: PERCENT. - [EM](): Higher is better; unit: PERCENT. ## Recorded results | Method | F1 | Result date | Report paper | | --- | --- | --- | --- | | GAPD | 93.8 | Not recorded | 6a191d82dbd7150506f82782 | | DecAF | 92.4 | Not recorded | 6a191d82dbd7150506f82782 | | KBQA-R1 | 91.5 | Not recorded | 6a191d82dbd7150506f82782 | | TIARA | 91.2 | Not recorded | 6a191d82dbd7150506f82782 | | RnG-KBQA | 89 | Not recorded | 6a191d82dbd7150506f82782 | | KBQA-o1 | 85.5 | Not recorded | 6a191d82dbd7150506f82782 | | KnowCoder-A1 | 81.1 | Not recorded | 6a191d82dbd7150506f82782 | | KBQA-R1 Harness | 70.1 | Not recorded | 6a191d82dbd7150506f82782 | | KBQA-R1 Harness | 68.6 | Not recorded | 6a191d82dbd7150506f82782 | | KBQA-R1 Harness | 64.3 | Not recorded | 6a191d82dbd7150506f82782 | | KBQA-R1 Harness | 62.1 | Not recorded | 6a191d82dbd7150506f82782 | | ARG-KBQA | 51.5 | Not recorded | 6a191d82dbd7150506f82782 | | KB-Coder | 45.5 | Not recorded | 6a191d82dbd7150506f82782 | | KB-BINDER | 43.3 | Not recorded | 6a191d82dbd7150506f82782 | ### GAPD F1: 93.8. Result ID: 6a1958e6fbc373592c13f2ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama-3.1-8B","Method Category":"Fine-tune-based Methods"}. This record does not assert independent reproduction. ### DecAF F1: 92.4. Result ID: 6a1958e6fbc373592c13f2a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"T5-large","Method Category":"Fine-tune-based Methods"}. This record does not assert independent reproduction. ### KBQA-R1 F1: 91.5. Result ID: 6a1958e6fbc373592c13f2ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama-3.1-8B","Method Category":"Fine-tune-based Methods"}. This record does not assert independent reproduction. ### TIARA F1: 91.2. Result ID: 6a1958e6fbc373592c13f2a8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"T5-large","Method Category":"Fine-tune-based Methods"}. This record does not assert independent reproduction. ### RnG-KBQA F1: 89. Result ID: 6a1958e6fbc373592c13f2a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"T5-large","Method Category":"Fine-tune-based Methods"}. This record does not assert independent reproduction. ### KBQA-o1 F1: 85.5. Result ID: 6a1958e6fbc373592c13f2a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama-3.1-8B","Method Category":"Fine-tune-based Methods"}. This record does not assert independent reproduction. ### KnowCoder-A1 F1: 81.1. Result ID: 6a1958e6fbc373592c13f2aa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Qwen2.5-Coder-7B","Method Category":"Fine-tune-based Methods"}. This record does not assert independent reproduction. ### KBQA-R1 Harness F1: 70.1. Result ID: 6a1958e6fbc373592c13f2a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Kimi-K2.5","Method Category":"Prompting Methods"}. This record does not assert independent reproduction. ### KBQA-R1 Harness F1: 68.6. Result ID: 6a1958e6fbc373592c13f2a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Claude 4.6 Sonnet","Method Category":"Prompting Methods"}. This record does not assert independent reproduction. ### KBQA-R1 Harness F1: 64.3. Result ID: 6a1958e6fbc373592c13f2a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"gpt-5.3-codex","Method Category":"Prompting Methods"}. This record does not assert independent reproduction. ### KBQA-R1 Harness F1: 62.1. Result ID: 6a1958e6fbc373592c13f2a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"GLM-5","Method Category":"Prompting Methods"}. This record does not assert independent reproduction. ### ARG-KBQA F1: 51.5. Result ID: 6a1958e6fbc373592c13f2a1. [Reporting paper]() Recorded evaluation configuration: {"LLM":"GPT-3.5-turbo","Method Category":"Prompting Methods"}. This record does not assert independent reproduction. ### KB-Coder F1: 45.5. Result ID: 6a1958e6fbc373592c13f2a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"GPT-3.5-turbo","Method Category":"Prompting Methods"}. This record does not assert independent reproduction. ### KB-BINDER F1: 43.3. Result ID: 6a1958e6fbc373592c13f29f. [Reporting paper]() Recorded evaluation configuration: {"LLM":"Codex-davinci-002","Method Category":"Prompting Methods"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.