# End-to-end RAG Generation on MS MARCO and Natural Questions

Source: [End-to-end RAG Generation on MS MARCO and Natural Questions](<https://www.sota2.com/research/sota/end-to-end-rag-generation-on-ms-marco-and-natural-questions>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-31T07:22:33.009Z

Evaluation of end-to-end Retrieval Augmented Generation (RAG) performance combining results from the MS MARCO and Natural Questions datasets, assessed using generation and QA metrics.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | End-to-end RAG generation |
| Dataset (including any stated split) | MS MARCO and Natural Questions |
| Metric | F1 (QA) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 7 of 7 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [F1 (QA)](<https://www.sota2.com/research/sota/end-to-end-rag-generation-on-ms-marco-and-natural-questions.md?metric=F1%20(QA)>): Higher is better; unit: PERCENT.

- [BLEU-4](<https://www.sota2.com/research/sota/end-to-end-rag-generation-on-ms-marco-and-natural-questions.md?metric=BLEU-4>): Higher is better; unit: PERCENT.

- [ROUGE-L](<https://www.sota2.com/research/sota/end-to-end-rag-generation-on-ms-marco-and-natural-questions.md?metric=ROUGE-L>): Higher is better; unit: PERCENT.

- [BERTScore](<https://www.sota2.com/research/sota/end-to-end-rag-generation-on-ms-marco-and-natural-questions.md?metric=BERTScore>): Higher is better; unit: PERCENT.

## Recorded results

| Method | F1 (QA) | Result date | Report paper |
| --- | --- | --- | --- |
| SPI | 50.1 | Not recorded | 6923e698d6b0dd334292f087 |
| Atlas | 48.3 | Not recorded | 6923e698d6b0dd334292f087 |
| Retro | 47.9 | Not recorded | 6923e698d6b0dd334292f087 |
| HyDE-RAG | 47.8 | Not recorded | 6923e698d6b0dd334292f087 |
| ColBERTv2-RAG | 47.2 | Not recorded | 6923e698d6b0dd334292f087 |
| SPLADE-RAG | 46.5 | Not recorded | 6923e698d6b0dd334292f087 |
| DPR-RAG | 44.8 | Not recorded | 6923e698d6b0dd334292f087 |

### SPI

F1 (QA): 50.1. Result ID: 69cb763447f34ce924b35ed0.

[Reporting paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

Recorded evaluation configuration: {"Latency (ms)":"92","Memory (GB)":"4.2"}. This record does not assert independent reproduction.

### Atlas

F1 (QA): 48.3. Result ID: 69cb763447f34ce924b35ece.

[Reporting paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6df71e209131cea3d024.md>)

[Implementation](<https://github.com/facebookresearch/atlas>)

Recorded evaluation configuration: {"Latency (ms)":"150","Memory (GB)":"7.5"}. This record does not assert independent reproduction.

### Retro

F1 (QA): 47.9. Result ID: 69cb763447f34ce924b35ecf.

[Reporting paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6bc41e209131cea302b0.md>)

Recorded evaluation configuration: {"Latency (ms)":"140","Memory (GB)":"6.8"}. This record does not assert independent reproduction.

### HyDE-RAG

F1 (QA): 47.8. Result ID: 69cb763447f34ce924b35ecd.

[Reporting paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

Recorded evaluation configuration: {"Latency (ms)":"155","Memory (GB)":"6.9"}. This record does not assert independent reproduction.

### ColBERTv2-RAG

F1 (QA): 47.2. Result ID: 69cb763447f34ce924b35ecc.

[Reporting paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

Recorded evaluation configuration: {"Latency (ms)":"128","Memory (GB)":"5.8"}. This record does not assert independent reproduction.

### SPLADE-RAG

F1 (QA): 46.5. Result ID: 69cb763447f34ce924b35ecb.

[Reporting paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

Recorded evaluation configuration: {"Latency (ms)":"115","Memory (GB)":"5.4"}. This record does not assert independent reproduction.

### DPR-RAG

F1 (QA): 44.8. Result ID: 69cb763447f34ce924b35eca.

[Reporting paper](<https://www.sota2.com/research/paper/6923e698d6b0dd334292f087.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa69b91e209131cea23c26.md>)

[Implementation](<https://github.com/huggingface/transformers/blob/master/examples/rag/>)

Recorded evaluation configuration: {"Latency (ms)":"145","Memory (GB)":"7.2"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.