# Single-hop Question Answering on Natural Questions (NQ) (test) Source: [Single-hop Question Answering on Natural Questions (NQ) (test)]() Collection index: [llms.txt]() Data updated: 2026-03-05T06:56:53.089Z Evaluation of model performance on the test split of the Natural Questions (NQ) dataset specifically for single-hop question answering, measured using EM and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Single-hop Question Answering | | Dataset (including any stated split) | Natural Questions (NQ) (test) | | Metric | EM | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 33 of 33 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [EM](): Higher is better; unit: PERCENT. - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | EM | Result date | Report paper | | --- | --- | --- | --- | | SE-Search-3B | 47.5 | Not recorded | 69a90ddc2d603e6de3e40587 | | AutoRefine-Base | 46.7 | Not recorded | 69a90ddc2d603e6de3e40587 | | O2-Searcher | 44.4 | Not recorded | 69a90ddc2d603e6de3e40587 | | AutoRefine-Instruct | 43.6 | Not recorded | 69a90ddc2d603e6de3e40587 | | ZeroSearch-Base | 43 | Not recorded | 69a90ddc2d603e6de3e40587 | | ReSearch-Base | 42.7 | Not recorded | 69a90ddc2d603e6de3e40587 | | Search-R1-Base | 42.1 | Not recorded | 69a90ddc2d603e6de3e40587 | | InForage | 42.1 | Not recorded | 69a90ddc2d603e6de3e40587 | | Search-R1-Instruct | 39.7 | Not recorded | 69a90ddc2d603e6de3e40587 | | ReSearch-Instruct | 36.5 | Not recorded | 69a90ddc2d603e6de3e40587 | | ProGraph-R1 | 35.94 | Not recorded | 69784684a2b28e761d87c4dc | | Naive RAG | 34.8 | Not recorded | 69a90ddc2d603e6de3e40587 | | ProGraph-R1 | 34.38 | Not recorded | 69784684a2b28e761d87c4dc | | Graph-R1 | 33.59 | Not recorded | 69784684a2b28e761d87c4dc | | Search-R1 | 32.03 | Not recorded | 69784684a2b28e761d87c4dc | | R1-Searcher | 32.03 | Not recorded | 69784684a2b28e761d87c4dc | | Graph-R1 | 30.47 | Not recorded | 69784684a2b28e761d87c4dc | | SFT | 24.9 | Not recorded | 69a90ddc2d603e6de3e40587 | | Search-R1 | 24.22 | Not recorded | 69784684a2b28e761d87c4dc | | R1-Searcher | 24.22 | Not recorded | 69784684a2b28e761d87c4dc | | Search-o1 | 23.8 | Not recorded | 69a90ddc2d603e6de3e40587 | | R1-Base | 22.6 | Not recorded | 69a90ddc2d603e6de3e40587 | | R1-Instruct | 21 | Not recorded | 69a90ddc2d603e6de3e40587 | | R1 | 16.41 | Not recorded | 69784684a2b28e761d87c4dc | | R1 | 11.72 | Not recorded | 69784684a2b28e761d87c4dc | | IRCoT | 11.1 | Not recorded | 69a90ddc2d603e6de3e40587 | | Direct Generation | 10.6 | Not recorded | 69a90ddc2d603e6de3e40587 | | SFT | 5.12 | Not recorded | 69784684a2b28e761d87c4dc | | SFT | 3.12 | Not recorded | 69784684a2b28e761d87c4dc | | NaiveGeneration | 2.34 | Not recorded | 69784684a2b28e761d87c4dc | | NaiveGeneration | 1.56 | Not recorded | 69784684a2b28e761d87c4dc | | StandardRAG | 1.56 | Not recorded | 69784684a2b28e761d87c4dc | | StandardRAG | 0 | Not recorded | 69784684a2b28e761d87c4dc | ### SE-Search-3B EM: 47.5. Result ID: 69a9292d6dbb7075b951fc1e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### AutoRefine-Base EM: 46.7. Result ID: 69a9292d6dbb7075b951fc1c. [Reporting paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### O2-Searcher EM: 44.4. Result ID: 69a9292d6dbb7075b951fc1a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### AutoRefine-Instruct EM: 43.6. Result ID: 69a9292d6dbb7075b951fc1b. [Reporting paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### ZeroSearch-Base EM: 43. Result ID: 69a9292d6dbb7075b951fc19. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### ReSearch-Base EM: 42.7. Result ID: 69a9292d6dbb7075b951fc18. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Search-R1-Base EM: 42.1. Result ID: 69a9292d6dbb7075b951fc16. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### InForage EM: 42.1. Result ID: 69a9292d6dbb7075b951fc1d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Search-R1-Instruct EM: 39.7. Result ID: 69a9292d6dbb7075b951fc15. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### ReSearch-Instruct EM: 36.5. Result ID: 69a9292d6dbb7075b951fc17. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### ProGraph-R1 EM: 35.94. Result ID: 69966f255698665ec4b61b72. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Graph-based knowledge"}. This record does not assert independent reproduction. ### Naive RAG EM: 34.8. Result ID: 69a9292d6dbb7075b951fc12. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Retrieval":"Workflow w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### ProGraph-R1 EM: 34.38. Result ID: 69966f255698665ec4b61b6a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Graph-based knowledge"}. This record does not assert independent reproduction. ### Graph-R1 EM: 33.59. Result ID: 69966f255698665ec4b61b71. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Graph-based knowledge"}. This record does not assert independent reproduction. ### Search-R1 EM: 32.03. Result ID: 69966f255698665ec4b61b6f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Chunk-based knowledge"}. This record does not assert independent reproduction. ### R1-Searcher EM: 32.03. Result ID: 69966f255698665ec4b61b70. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Chunk-based knowledge"}. This record does not assert independent reproduction. ### Graph-R1 EM: 30.47. Result ID: 69966f255698665ec4b61b69. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Graph-based knowledge"}. This record does not assert independent reproduction. ### SFT EM: 24.9. Result ID: 69a9292d6dbb7075b951fc0f. [Reporting paper]() Recorded evaluation configuration: {"Retrieval":"w/o Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Search-R1 EM: 24.22. Result ID: 69966f255698665ec4b61b67. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Chunk-based knowledge"}. This record does not assert independent reproduction. ### R1-Searcher EM: 24.22. Result ID: 69966f255698665ec4b61b68. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Training","Knowledge Interaction":"Chunk-based knowledge"}. This record does not assert independent reproduction. ### Search-o1 EM: 23.8. Result ID: 69a9292d6dbb7075b951fc14. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Retrieval":"Agent w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### R1-Base EM: 22.6. Result ID: 69a9292d6dbb7075b951fc11. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"w/o Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### R1-Instruct EM: 21. Result ID: 69a9292d6dbb7075b951fc10. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Retrieval":"w/o Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### R1 EM: 16.41. Result ID: 69966f255698665ec4b61b6e. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Training","Knowledge Interaction":"No knowledge interaction"}. This record does not assert independent reproduction. ### R1 EM: 11.72. Result ID: 69966f255698665ec4b61b66. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Training","Knowledge Interaction":"No knowledge interaction"}. This record does not assert independent reproduction. ### IRCoT EM: 11.1. Result ID: 69a9292d6dbb7075b951fc13. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Retrieval":"Workflow w/ Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### Direct Generation EM: 10.6. Result ID: 69a9292d6dbb7075b951fc0e. [Reporting paper]() Recorded evaluation configuration: {"Retrieval":"w/o Retrieval","Backbone":"Qwen2.5-3B"}. This record does not assert independent reproduction. ### SFT EM: 5.12. Result ID: 69966f255698665ec4b61b6d. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Training","Knowledge Interaction":"No knowledge interaction"}. This record does not assert independent reproduction. ### SFT EM: 3.12. Result ID: 69966f255698665ec4b61b65. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Training","Knowledge Interaction":"No knowledge interaction"}. This record does not assert independent reproduction. ### NaiveGeneration EM: 2.34. Result ID: 69966f255698665ec4b61b63. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Prompt engineering","Knowledge Interaction":"No knowledge interaction"}. This record does not assert independent reproduction. ### NaiveGeneration EM: 1.56. Result ID: 69966f255698665ec4b61b6b. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Prompt engineering","Knowledge Interaction":"No knowledge interaction"}. This record does not assert independent reproduction. ### StandardRAG EM: 1.56. Result ID: 69966f255698665ec4b61b6c. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B-Instruct","Optimization Method":"Prompt engineering","Knowledge Interaction":"Chunk-based knowledge"}. This record does not assert independent reproduction. ### StandardRAG EM: 0. Result ID: 69966f255698665ec4b61b64. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Optimization Method":"Prompt engineering","Knowledge Interaction":"Chunk-based knowledge"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.