# Open-Domain Question Answering on WQ (test)

Source: [Open-Domain Question Answering on WQ (test)](<https://www.sota2.com/research/sota/open-domain-question-answering-on-wq-test>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:41:36.796Z

Evaluation of model performance on the test split of the WebQuestions (WQ) dataset for Open-Domain Question Answering, measured using Exact Match (EM) and F1 score.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Open-Domain Question Answering |
| Dataset (including any stated split) | WQ (test) |
| Metric | EM |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 38 of 38 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [EM](<https://www.sota2.com/research/sota/open-domain-question-answering-on-wq-test.md?metric=EM>): Higher is better; unit: PERCENT.

- [F1 Score](<https://www.sota2.com/research/sota/open-domain-question-answering-on-wq-test.md?metric=F1%20Score>): Higher is better; unit: PERCENT.

- [Token Count](<https://www.sota2.com/research/sota/open-domain-question-answering-on-wq-test.md?metric=Token%20Count>): Higher is better; unit: SCALAR.

## Recorded results

| Method | EM | Result date | Report paper |
| --- | --- | --- | --- |
| ECoRAG | 33.71 | Not recorded | 68fa792a1e209131cea7c5aa |
| CompAct | 33.07 | Not recorded | 68fa792a1e209131cea7c5aa |
| closed-book | 32.97 | Not recorded | 68fa792a1e209131cea7c5aa |
| LongLLMLingua | 32.78 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (extractive) | 31.25 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (abstractive) | 31.15 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua | 31.1 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua-2 | 30.61 | Not recorded | 68fa792a1e209131cea7c5aa |
| ECoRAG | 30.17 | Not recorded | 68fa792a1e209131cea7c5aa |
| ECoRAG | 30.17 | Not recorded | 68fa792a1e209131cea7c5aa |
| CompAct | 29.77 | Not recorded | 68fa792a1e209131cea7c5aa |
| ECoRAG | 27.41 | Not recorded | 68fa792a1e209131cea7c5aa |
| LongLLMLingua | 27.02 | Not recorded | 68fa792a1e209131cea7c5aa |
| closed-book | 26.77 | Not recorded | 68fa792a1e209131cea7c5aa |
| CompAct | 26.67 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (extractive) | 26.03 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (abstractive) | 25.2 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua | 25.04 | Not recorded | 68fa792a1e209131cea7c5aa |
| closed-book | 24.51 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua-2 | 24.36 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua-2 | 23.52 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua | 22.98 | Not recorded | 68fa792a1e209131cea7c5aa |
| closed-book | 21.79 | Not recorded | 68fa792a1e209131cea7c5aa |
| ECoRAG | 21.6 | Not recorded | 68fa792a1e209131cea7c5aa |
| LongLLMLingua | 21.51 | Not recorded | 68fa792a1e209131cea7c5aa |
| CompAct | 21.41 | Not recorded | 68fa792a1e209131cea7c5aa |
| standard RAG (100 documents) | 21.11 | Not recorded | 68fa792a1e209131cea7c5aa |
| BGE-M3 | 20.77 | Not recorded | 68fa792a1e209131cea7c5aa |
| BGE-M3 | 20.77 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (abstractive) | 20.47 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (extractive) | 19.54 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (abstractive) | 18.8 | Not recorded | 68fa792a1e209131cea7c5aa |
| LongLLMLingua | 18.7 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua | 17.62 | Not recorded | 68fa792a1e209131cea7c5aa |
| RECOMP (extractive) | 16.19 | Not recorded | 68fa792a1e209131cea7c5aa |
| LLMLingua-2 | 4.28 | Not recorded | 68fa792a1e209131cea7c5aa |
| standard RAG | 0.25 | Not recorded | 68fa792a1e209131cea7c5aa |

### ECoRAG

EM: 33.71. Result ID: 6995cb413badfaffe6cf789a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Implementation](<https://github.com/ldilab/ECoRAG>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":671}. This record does not assert independent reproduction.

### CompAct

EM: 33.07. Result ID: 6995cb413badfaffe6cf7899.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75381e209131cea6689e.md>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":218}. This record does not assert independent reproduction.

### closed-book

EM: 32.97. Result ID: 6995cb413badfaffe6cf7893.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":0}. This record does not assert independent reproduction.

### LongLLMLingua

EM: 32.78. Result ID: 6995cb413badfaffe6cf7896.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61c6d.md>)

[Implementation](<https://aka.ms/LLMLingua>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":1629}. This record does not assert independent reproduction.

### RECOMP (extractive)

EM: 31.25. Result ID: 6995cb413badfaffe6cf7897.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":857}. This record does not assert independent reproduction.

### RECOMP (abstractive)

EM: 31.15. Result ID: 6995cb413badfaffe6cf7898.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":157}. This record does not assert independent reproduction.

### LLMLingua

EM: 31.1. Result ID: 6995cb413badfaffe6cf7894.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71621e209131cea5103a.md>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":868}. This record does not assert independent reproduction.

### LLMLingua-2

EM: 30.61. Result ID: 6995cb413badfaffe6cf7895.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61cb2.md>)

[Implementation](<https://aka.ms/LLMLingua-2>)

Recorded evaluation configuration: {"Reader Model":"Flan-UL2","#tokens":1580}. This record does not assert independent reproduction.

### ECoRAG

EM: 30.17. Result ID: 6995cb413badfaffe6cf78ae.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Implementation](<https://github.com/ldilab/ECoRAG>)

Recorded evaluation configuration: {"#tokens":560,"Backbone":"GPT-4o-mini"}. This record does not assert independent reproduction.

### ECoRAG

EM: 30.17. Result ID: 6995cb413badfaffe6cf78c3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Implementation](<https://github.com/ldilab/ECoRAG>)

Recorded evaluation configuration: {"#tokens":560}. This record does not assert independent reproduction.

### CompAct

EM: 29.77. Result ID: 6995cb413badfaffe6cf78c2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75381e209131cea6689e.md>)

Recorded evaluation configuration: {"#tokens":75}. This record does not assert independent reproduction.

### ECoRAG

EM: 27.41. Result ID: 6995cb413badfaffe6cf78ec.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Implementation](<https://github.com/ldilab/ECoRAG>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression strategy":"RAG with 100 documents compressed"}. This record does not assert independent reproduction.

### LongLLMLingua

EM: 27.02. Result ID: 6995cb413badfaffe6cf78e8.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61c6d.md>)

[Implementation](<https://aka.ms/LLMLingua>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression strategy":"RAG with 100 documents compressed"}. This record does not assert independent reproduction.

### closed-book

EM: 26.77. Result ID: 6995cb413badfaffe6cf78e4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression":"None"}. This record does not assert independent reproduction.

### CompAct

EM: 26.67. Result ID: 6995cb413badfaffe6cf78eb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75381e209131cea6689e.md>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression strategy":"RAG with 100 documents compressed"}. This record does not assert independent reproduction.

### RECOMP (extractive)

EM: 26.03. Result ID: 6995cb413badfaffe6cf78e9.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression strategy":"RAG with 100 documents compressed"}. This record does not assert independent reproduction.

### RECOMP (abstractive)

EM: 25.2. Result ID: 6995cb413badfaffe6cf78ea.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression strategy":"RAG with 100 documents compressed"}. This record does not assert independent reproduction.

### LLMLingua

EM: 25.04. Result ID: 6995cb413badfaffe6cf78e6.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71621e209131cea5103a.md>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression strategy":"RAG with 100 documents compressed"}. This record does not assert independent reproduction.

### closed-book

EM: 24.51. Result ID: 6995cb413badfaffe6cf78bb.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

Recorded evaluation configuration: {"#tokens":0}. This record does not assert independent reproduction.

### LLMLingua-2

EM: 24.36. Result ID: 6995cb413badfaffe6cf78e7.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61cb2.md>)

[Implementation](<https://aka.ms/LLMLingua-2>)

Recorded evaluation configuration: {"Reader model":"Gemma2","Compression strategy":"RAG with 100 documents compressed"}. This record does not assert independent reproduction.

### LLMLingua-2

EM: 23.52. Result ID: 6995cb413badfaffe6cf78be.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61cb2.md>)

[Implementation](<https://aka.ms/LLMLingua-2>)

Recorded evaluation configuration: {"#tokens":1113}. This record does not assert independent reproduction.

### LLMLingua

EM: 22.98. Result ID: 6995cb413badfaffe6cf78bd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71621e209131cea5103a.md>)

Recorded evaluation configuration: {"#tokens":641}. This record does not assert independent reproduction.

### closed-book

EM: 21.79. Result ID: 6995cb413badfaffe6cf78cd.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":0,"Compression":"None"}. This record does not assert independent reproduction.

### ECoRAG

EM: 21.6. Result ID: 6995cb413badfaffe6cf78d5.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Implementation](<https://github.com/ldilab/ECoRAG>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"Compressed"}. This record does not assert independent reproduction.

### LongLLMLingua

EM: 21.51. Result ID: 6995cb413badfaffe6cf78bf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61c6d.md>)

[Implementation](<https://aka.ms/LLMLingua>)

Recorded evaluation configuration: {"#tokens":1355}. This record does not assert independent reproduction.

### CompAct

EM: 21.41. Result ID: 6995cb413badfaffe6cf78d4.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa75381e209131cea6689e.md>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"Compressed"}. This record does not assert independent reproduction.

### standard RAG (100 documents)

EM: 21.11. Result ID: 6995cb413badfaffe6cf78bc.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa686d1e209131cea1c190.md>)

[Implementation](<https://github.com/facebookresearch/DPR>)

Recorded evaluation configuration: {"#tokens":13731}. This record does not assert independent reproduction.

### BGE-M3

EM: 20.77. Result ID: 6995cb413badfaffe6cf78ac.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73d91e209131cea5e7ab.md>)

[Implementation](<https://github.com/FlagOpen/FlagEmbedding>)

Recorded evaluation configuration: {"Top-k retrieval":10,"#tokens":322,"Backbone":"GPT-4o-mini"}. This record does not assert independent reproduction.

### BGE-M3

EM: 20.77. Result ID: 6995cb413badfaffe6cf78ad.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73d91e209131cea5e7ab.md>)

[Implementation](<https://github.com/FlagOpen/FlagEmbedding>)

Recorded evaluation configuration: {"Top-k retrieval":20,"#tokens":645,"Backbone":"GPT-4o-mini"}. This record does not assert independent reproduction.

### RECOMP (abstractive)

EM: 20.47. Result ID: 6995cb413badfaffe6cf78c1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"#tokens":19}. This record does not assert independent reproduction.

### RECOMP (extractive)

EM: 19.54. Result ID: 6995cb413badfaffe6cf78c0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"#tokens":658}. This record does not assert independent reproduction.

### RECOMP (abstractive)

EM: 18.8. Result ID: 6995cb413badfaffe6cf78d3.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"Compressed"}. This record does not assert independent reproduction.

### LongLLMLingua

EM: 18.7. Result ID: 6995cb413badfaffe6cf78d1.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61c6d.md>)

[Implementation](<https://aka.ms/LLMLingua>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"Compressed"}. This record does not assert independent reproduction.

### LLMLingua

EM: 17.62. Result ID: 6995cb413badfaffe6cf78cf.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa71621e209131cea5103a.md>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"Compressed"}. This record does not assert independent reproduction.

### RECOMP (extractive)

EM: 16.19. Result ID: 6995cb413badfaffe6cf78d2.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70c81e209131cea4d0a5.md>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"Compressed"}. This record does not assert independent reproduction.

### LLMLingua-2

EM: 4.28. Result ID: 6995cb413badfaffe6cf78d0.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa74701e209131cea61cb2.md>)

[Implementation](<https://aka.ms/LLMLingua-2>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"Compressed"}. This record does not assert independent reproduction.

### standard RAG

EM: 0.25. Result ID: 6995cb413badfaffe6cf78ce.

[Reporting paper](<https://www.sota2.com/research/paper/68fa792a1e209131cea7c5aa.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa686d1e209131cea1c190.md>)

[Implementation](<https://github.com/facebookresearch/DPR>)

Recorded evaluation configuration: {"Reader Model":"Llama3","Retrieved Documents":100,"Compression":"None"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.