# General Question Answering on NQ (Natural Questions) (EM, F1aware)

Source: [General Question Answering on NQ (Natural Questions) (EM, F1aware)](<https://www.sota2.com/research/sota/general-question-answering-on-nq-natural-questions-em-f1aware>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-21T07:19:04.649Z

Evaluation of model performance on the Natural Questions (NQ) dataset for General Question Answering, measured using Exact Match (EM) and F1aware scores.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | General Question Answering |
| Dataset (including any stated split) | NQ (Natural Questions) |
| Metric | EM |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 32 of 32 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [EM](<https://www.sota2.com/research/sota/general-question-answering-on-nq-natural-questions-em-f1aware.md?metric=EM>): Higher is better; unit: PERCENT.

- [F1aware](<https://www.sota2.com/research/sota/general-question-answering-on-nq-natural-questions-em-f1aware.md?metric=F1aware>): Higher is better; unit: PERCENT.

## Recorded results

| Method | EM | Result date | Report paper |
| --- | --- | --- | --- |
| Search-R1 | 46.1 | Not recorded | 6944dc29423ea42265729cf9 |
| ADASEARCH | 43.5 | Not recorded | 6944dc29423ea42265729cf9 |
| Search-R1 | 42.7 | Not recorded | 6944dc29423ea42265729cf9 |
| GraphRAG-Router | 42.6 | Not recorded | 69e704887d0172140ec7ec6a |
| IKEA | 42.2 | Not recorded | 6944dc29423ea42265729cf9 |
| IKEA | 41.4 | Not recorded | 6944dc29423ea42265729cf9 |
| Awareness Shaping | 41.4 | Not recorded | 6944dc29423ea42265729cf9 |
| Naive Shaping | 41.2 | Not recorded | 6944dc29423ea42265729cf9 |
| Search-R1 | 40.3 | Not recorded | 69e704887d0172140ec7ec6a |
| Router-R1 | 38.6 | Not recorded | 69e704887d0172140ec7ec6a |
| RL w/o search | 38.2 | Not recorded | 6944dc29423ea42265729cf9 |
| ADASEARCH | 37.9 | Not recorded | 6944dc29423ea42265729cf9 |
| Naive Shaping | 37.8 | Not recorded | 6944dc29423ea42265729cf9 |
| Awareness Shaping | 37.7 | Not recorded | 6944dc29423ea42265729cf9 |
| GraphSearch | 36.8 | Not recorded | 69e704887d0172140ec7ec6a |
| HippoRAG2 | 35.3 | Not recorded | 69e704887d0172140ec7ec6a |
| Search-o1 | 34.8 | Not recorded | 69e704887d0172140ec7ec6a |
| RAG | 31.7 | Not recorded | 6944dc29423ea42265729cf9 |
| LinearRAG | 30.2 | Not recorded | 69e704887d0172140ec7ec6a |
| GraphRAG | 30.1 | Not recorded | 69e704887d0172140ec7ec6a |
| RAG | 29.4 | Not recorded | 6944dc29423ea42265729cf9 |
| Vanilla RAG | 29.4 | Not recorded | 69e704887d0172140ec7ec6a |
| CoT | 28.6 | Not recorded | 69e704887d0172140ec7ec6a |
| RAPTOR | 27.1 | Not recorded | 69e704887d0172140ec7ec6a |
| HyperGraphRAG | 26.8 | Not recorded | 69e704887d0172140ec7ec6a |
| Graph-R1 | 26.7 | Not recorded | 69e704887d0172140ec7ec6a |
| CoT | 25 | Not recorded | 6944dc29423ea42265729cf9 |
| RL w/o search | 21.7 | Not recorded | 6944dc29423ea42265729cf9 |
| Direct Answer | 18.8 | Not recorded | 6944dc29423ea42265729cf9 |
| CoT | 14.1 | Not recorded | 6944dc29423ea42265729cf9 |
| Direct Infer | 10.7 | Not recorded | 69e704887d0172140ec7ec6a |
| Direct Answer | 8.1 | Not recorded | 6944dc29423ea42265729cf9 |

### Search-R1

EM: 46.1. Result ID: 6996b9bb5698665ec40471bc.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a651e209131cea82e7b.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Outcome RL Baselines"}. This record does not assert independent reproduction.

### ADASEARCH

EM: 43.5. Result ID: 6996b9bb5698665ec40471c0.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Paper Method"}. This record does not assert independent reproduction.

### Search-R1

EM: 42.7. Result ID: 6996b9bb5698665ec40471b3.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a651e209131cea82e7b.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Outcome RL Baselines"}. This record does not assert independent reproduction.

### GraphRAG-Router

EM: 42.6. Result ID: 69e724e647f34ce924cced8c.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

Recorded evaluation configuration: {"System":"Our Method"}. This record does not assert independent reproduction.

### IKEA

EM: 42.2. Result ID: 6996b9bb5698665ec40471bf.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa787a1e209131cea78943.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Reward-Shaping Baselines"}. This record does not assert independent reproduction.

### IKEA

EM: 41.4. Result ID: 6996b9bb5698665ec40471b6.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa787a1e209131cea78943.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Reward-Shaping Baselines"}. This record does not assert independent reproduction.

### Awareness Shaping

EM: 41.4. Result ID: 6996b9bb5698665ec40471be.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Reward-Shaping Baselines"}. This record does not assert independent reproduction.

### Naive Shaping

EM: 41.2. Result ID: 6996b9bb5698665ec40471bd.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Reward-Shaping Baselines"}. This record does not assert independent reproduction.

### Search-R1

EM: 40.3. Result ID: 69e724e647f34ce924cced89.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a651e209131cea82e7b.md>)

Recorded evaluation configuration: {"System":"RL-based Agent"}. This record does not assert independent reproduction.

### Router-R1

EM: 38.6. Result ID: 69e724e647f34ce924cced8b.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa797c1e209131cea7dc45.md>)

Recorded evaluation configuration: {"System":"RL-based Agent"}. This record does not assert independent reproduction.

### RL w/o search

EM: 38.2. Result ID: 6996b9bb5698665ec40471bb.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a651e209131cea82e7b.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Outcome RL Baselines"}. This record does not assert independent reproduction.

### ADASEARCH

EM: 37.9. Result ID: 6996b9bb5698665ec40471b7.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Paper Method"}. This record does not assert independent reproduction.

### Naive Shaping

EM: 37.8. Result ID: 6996b9bb5698665ec40471b4.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Reward-Shaping Baselines"}. This record does not assert independent reproduction.

### Awareness Shaping

EM: 37.7. Result ID: 6996b9bb5698665ec40471b5.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Reward-Shaping Baselines"}. This record does not assert independent reproduction.

### GraphSearch

EM: 36.8. Result ID: 69e724e647f34ce924cced88.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bc71e209131cea8a683.md>)

Recorded evaluation configuration: {"System":"Training-free Agent"}. This record does not assert independent reproduction.

### HippoRAG2

EM: 35.3. Result ID: 69e724e647f34ce924cced82.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79821e209131cea7de8a.md>)

Recorded evaluation configuration: {"System":"RAG-Based"}. This record does not assert independent reproduction.

### Search-o1

EM: 34.8. Result ID: 69e724e647f34ce924cced87.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76911e209131cea6dca9.md>)

[Implementation](<https://github.com/sunnynexus/Search-o1>)

Recorded evaluation configuration: {"System":"Training-free Agent"}. This record does not assert independent reproduction.

### RAG

EM: 31.7. Result ID: 6996b9bb5698665ec40471b1.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Prompting Baselines"}. This record does not assert independent reproduction.

### LinearRAG

EM: 30.2. Result ID: 69e724e647f34ce924cced83.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c131e209131cea8c4cc.md>)

Recorded evaluation configuration: {"System":"RAG-Based"}. This record does not assert independent reproduction.

### GraphRAG

EM: 30.1. Result ID: 69e724e647f34ce924cced86.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa770d1e209131cea70c5a.md>)

Recorded evaluation configuration: {"System":"RAG-Based"}. This record does not assert independent reproduction.

### RAG

EM: 29.4. Result ID: 6996b9bb5698665ec40471ba.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Prompting Baselines"}. This record does not assert independent reproduction.

### Vanilla RAG

EM: 29.4. Result ID: 69e724e647f34ce924cced81.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"System":"RAG-Based"}. This record does not assert independent reproduction.

### CoT

EM: 28.6. Result ID: 69e724e647f34ce924cced80.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e571e209131cea3f27b.md>)

Recorded evaluation configuration: {"System":"Basic LLM"}. This record does not assert independent reproduction.

### RAPTOR

EM: 27.1. Result ID: 69e724e647f34ce924cced85.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71fe1e209131cea53fc2.md>)

Recorded evaluation configuration: {"System":"RAG-Based"}. This record does not assert independent reproduction.

### HyperGraphRAG

EM: 26.8. Result ID: 69e724e647f34ce924cced84.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c4a1e209131cea8d698.md>)

Recorded evaluation configuration: {"System":"RAG-Based"}. This record does not assert independent reproduction.

### Graph-R1

EM: 26.7. Result ID: 69e724e647f34ce924cced8a.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a431e209131cea82033.md>)

Recorded evaluation configuration: {"System":"RL-based Agent"}. This record does not assert independent reproduction.

### CoT

EM: 25. Result ID: 6996b9bb5698665ec40471b9.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e571e209131cea3f27b.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Prompting Baselines"}. This record does not assert independent reproduction.

### RL w/o search

EM: 21.7. Result ID: 6996b9bb5698665ec40471b2.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7a651e209131cea82e7b.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Outcome RL Baselines"}. This record does not assert independent reproduction.

### Direct Answer

EM: 18.8. Result ID: 6996b9bb5698665ec40471b8.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","Category":"Prompting Baselines"}. This record does not assert independent reproduction.

### CoT

EM: 14.1. Result ID: 6996b9bb5698665ec40471b0.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e571e209131cea3f27b.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Prompting Baselines"}. This record does not assert independent reproduction.

### Direct Infer

EM: 10.7. Result ID: 69e724e647f34ce924cced7f.

[Reporting paper](<https://www.sota2.com/research/paper/69e704887d0172140ec7ec6a.md>)

Recorded evaluation configuration: {"System":"Basic LLM"}. This record does not assert independent reproduction.

### Direct Answer

EM: 8.1. Result ID: 6996b9bb5698665ec40471af.

[Reporting paper](<https://www.sota2.com/research/paper/6944dc29423ea42265729cf9.md>)

Recorded evaluation configuration: {"Base Model":"Qwen2.5-3B-Instruct","Category":"Prompting Baselines"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.