Question Answering on HotpotQA December 2018 Wikipedia dump (test)
37.23EMDoc-Search
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Doc-SearchRetriever=E5-base-v2, Knowledge Base=Wikipedia-182026.05 | 37.23 | — | — | |
| EXITReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 37 | 48.3 | 3.3 | |
| StepSearchRetriever=E5-base-v2, Knowledge Base=Wikipedia-182026.05 | 36.31 | — | — | |
| Search-R1Retriever=E5-base-v2, Knowledge Base=Wikipedia-182026.05 | 36.16 | — | — | |
| ReSearchRetriever=E5-base-v2, Knowledge Base=Wikipedia-182026.05 | 34.88 | — | — | |
| DeepResearcherRetriever=E5-base-v2, Knowledge Base=Wikipedia-182026.05 | 34.85 | — | — | |
| LongLLMLinguaReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 34.1 | 45.3 | 4 | |
| CompActReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 33.8 | 44.1 | 11 | |
| RefinerReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 33.8 | 44.7 | 14.6 | |
| Original DocsReader Model=Llama3.1-70B-Instruct, Compressor Type=None2024.12 | 33.7 | 44.5 | 8.3 | |
| R-SearchRetriever=E5-base-v2, Knowledge Base=Wikipedia-182026.05 | 32.59 | — | — | |
| ZeroSearchRetriever=E5-base-v2, Knowledge Base=Wikipedia-182026.05 | 32.11 | — | — | |
| EXITReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 30.6 | 41.5 | 0.8 | |
| RECOMP-AbstReader Model=Llama3.1-70B-Instruct, Compressor Type=Abstractive2024.12 | 30.3 | 40.8 | 4.4 | |
| CompActReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 28.8 | 39.8 | 8.3 | |
| RefinerReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 28.8 | 40 | 6.9 | |
| Original DocsReader Model=Llama3.1-8B-Instruct, Compressor Type=None2024.12 | 28.1 | 38.6 | 1 | |
| LongLLMLinguaReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 28 | 38 | 0.8 | |
| RECOMP-ExtrReader Model=Llama3.1-70B-Instruct, Compressor Type=Extractive2024.12 | 27.6 | 36.7 | 2.9 | |
| RECOMP-AbstReader Model=Llama3.1-8B-Instruct, Compressor Type=Abstractive2024.12 | 26.5 | 37 | 2.2 | |
| RECOMP-ExtrReader Model=Llama3.1-8B-Instruct, Compressor Type=Extractive2024.12 | 23.4 | 32.8 | 0.4 |