# Question Answering on Natural Questions (NQ) (EM and F1 Metrics) Source: [Question Answering on Natural Questions (NQ) (EM and F1 Metrics)]() Collection index: [llms.txt]() Data updated: 2026-05-27T08:04:02.744Z Evaluation of model performance on the Natural Questions (NQ) dataset for Question Answering, specifically reporting Exact Match (EM) and F1 scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | Natural Questions (NQ) | | Metric | Exact Match (EM) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Exact Match (EM)](): Higher is better; unit: PERCENT. - [F1 Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Exact Match (EM) | Result date | Report paper | | --- | --- | --- | --- | | RAG-GD | 45.68 | Not recorded | 6a167aa7dbd7150506f80070 | | Base adapter | 43.46 | Not recorded | 6a167aa7dbd7150506f80070 | | RAG-GD | 42.91 | Not recorded | 6a167aa7dbd7150506f80070 | | Base adapter | 41.77 | Not recorded | 6a167aa7dbd7150506f80070 | | Vanilla RAG | 40.72 | Not recorded | 6a167aa7dbd7150506f80070 | | RAG-GD | 40.22 | Not recorded | 6a167aa7dbd7150506f80070 | | Vanilla RAG | 39.16 | Not recorded | 6a167aa7dbd7150506f80070 | | Base adapter | 38.47 | Not recorded | 6a167aa7dbd7150506f80070 | | SAVER | 37.1 | Not recorded | 69d8843d7d0172140ec77c26 | | SAVER | 36.8 | Not recorded | 69d8843d7d0172140ec77c26 | | MAD | 36.6 | Not recorded | 69d8843d7d0172140ec77c26 | | MAD | 36.2 | Not recorded | 69d8843d7d0172140ec77c26 | | B-2 | 35.9 | Not recorded | 69d8843d7d0172140ec77c26 | | B-2 | 35.5 | Not recorded | 69d8843d7d0172140ec77c26 | | Self-Refine | 34.6 | Not recorded | 69d8843d7d0172140ec77c26 | | RAG-GD | 34.46 | Not recorded | 6a167aa7dbd7150506f80070 | | B-2 | 34.4 | Not recorded | 69d8843d7d0172140ec77c26 | | SAVER | 33.9 | Not recorded | 69d8843d7d0172140ec77c26 | | Self-Refine | 33.5 | Not recorded | 69d8843d7d0172140ec77c26 | | MAD | 33.4 | Not recorded | 69d8843d7d0172140ec77c26 | | CoT | 33 | Not recorded | 69d8843d7d0172140ec77c26 | | CoT | 32.7 | Not recorded | 69d8843d7d0172140ec77c26 | | Base adapter | 32.57 | Not recorded | 6a167aa7dbd7150506f80070 | | Vanilla RAG | 31.41 | Not recorded | 6a167aa7dbd7150506f80070 | | Self-Refine | 31.2 | Not recorded | 69d8843d7d0172140ec77c26 | | Vanilla LM | 29.5 | Not recorded | 69d8843d7d0172140ec77c26 | | CoT | 29.5 | Not recorded | 69d8843d7d0172140ec77c26 | | Vanilla LM | 28.2 | Not recorded | 69d8843d7d0172140ec77c26 | | Vanilla RAG | 27.61 | Not recorded | 6a167aa7dbd7150506f80070 | | Vanilla LM | 25.1 | Not recorded | 69d8843d7d0172140ec77c26 | | Query Only | 22.46 | Not recorded | 6a167aa7dbd7150506f80070 | | Query Only | 15.95 | Not recorded | 6a167aa7dbd7150506f80070 | ### RAG-GD Exact Match (EM): 45.68. Result ID: 6a16a56efbc373592c0fa25d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Retriever":"E5","K":"5"}. This record does not assert independent reproduction. ### Base adapter Exact Match (EM): 43.46. Result ID: 6a16a56efbc373592c0fa25b. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Retriever":"E5"}. This record does not assert independent reproduction. ### RAG-GD Exact Match (EM): 42.91. Result ID: 6a16a56efbc373592c0fa256. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Retriever":"E5","K":"5"}. This record does not assert independent reproduction. ### Base adapter Exact Match (EM): 41.77. Result ID: 6a16a56efbc373592c0fa254. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Retriever":"E5"}. This record does not assert independent reproduction. ### Vanilla RAG Exact Match (EM): 40.72. Result ID: 6a16a56efbc373592c0fa259. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Retriever":"E5"}. This record does not assert independent reproduction. ### RAG-GD Exact Match (EM): 40.22. Result ID: 6a16a56efbc373592c0fa25c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Retriever":"BM25","K":"5"}. This record does not assert independent reproduction. ### Vanilla RAG Exact Match (EM): 39.16. Result ID: 6a16a56efbc373592c0fa252. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Retriever":"E5"}. This record does not assert independent reproduction. ### Base adapter Exact Match (EM): 38.47. Result ID: 6a16a56efbc373592c0fa25a. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Retriever":"BM25"}. This record does not assert independent reproduction. ### SAVER Exact Match (EM): 37.1. Result ID: 69d8c48647f34ce924c208a9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### SAVER Exact Match (EM): 36.8. Result ID: 69d8c48647f34ce924c208b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### MAD Exact Match (EM): 36.6. Result ID: 69d8c48647f34ce924c208a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### MAD Exact Match (EM): 36.2. Result ID: 69d8c48647f34ce924c208b2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### B-2 Exact Match (EM): 35.9. Result ID: 69d8c48647f34ce924c208a8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### B-2 Exact Match (EM): 35.5. Result ID: 69d8c48647f34ce924c208b4. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### Self-Refine Exact Match (EM): 34.6. Result ID: 69d8c48647f34ce924c208b3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### RAG-GD Exact Match (EM): 34.46. Result ID: 6a16a56efbc373592c0fa255. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Retriever":"BM25","K":"5"}. This record does not assert independent reproduction. ### B-2 Exact Match (EM): 34.4. Result ID: 69d8c48647f34ce924c208ae. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### SAVER Exact Match (EM): 33.9. Result ID: 69d8c48647f34ce924c208af. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### Self-Refine Exact Match (EM): 33.5. Result ID: 69d8c48647f34ce924c208a7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### MAD Exact Match (EM): 33.4. Result ID: 69d8c48647f34ce924c208ac. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### CoT Exact Match (EM): 33. Result ID: 69d8c48647f34ce924c208b1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### CoT Exact Match (EM): 32.7. Result ID: 69d8c48647f34ce924c208a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### Base adapter Exact Match (EM): 32.57. Result ID: 6a16a56efbc373592c0fa253. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Retriever":"BM25"}. This record does not assert independent reproduction. ### Vanilla RAG Exact Match (EM): 31.41. Result ID: 6a16a56efbc373592c0fa258. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Retriever":"BM25"}. This record does not assert independent reproduction. ### Self-Refine Exact Match (EM): 31.2. Result ID: 69d8c48647f34ce924c208ad. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### Vanilla LM Exact Match (EM): 29.5. Result ID: 69d8c48647f34ce924c208a4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.1-8B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### CoT Exact Match (EM): 29.5. Result ID: 69d8c48647f34ce924c208ab. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### Vanilla LM Exact Match (EM): 28.2. Result ID: 69d8c48647f34ce924c208b0. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### Vanilla RAG Exact Match (EM): 27.61. Result ID: 6a16a56efbc373592c0fa251. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Retriever":"BM25"}. This record does not assert independent reproduction. ### Vanilla LM Exact Match (EM): 25.1. Result ID: 69d8c48647f34ce924c208aa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaMA-3.2-3B","Inference Setting":"Zero-shot"}. This record does not assert independent reproduction. ### Query Only Exact Match (EM): 22.46. Result ID: 6a16a56efbc373592c0fa257. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Retriever":"None"}. This record does not assert independent reproduction. ### Query Only Exact Match (EM): 15.95. Result ID: 6a16a56efbc373592c0fa250. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5-7B","Retriever":"None"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.