Multi-hop Question Answering on HotpotQA fullwiki setting (dev)
81.5Answer F1PATHFID+
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| PATHFID+Top-K paths=Top-25 paths, Evaluation Protocol=On Dev* (Oracle expanded passages)2022.05 | 81.5 | 70.2 | — | — | 60.9 | 86.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DOCHOPPERreranking=true2021.06 | 79.7 | 66.5 | — | — | 61.4 | 85.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HopRetriever-plusRetriever=MDR2022.05 | 79.2 | 66.6 | — | — | 56 | 81.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IDRQA2020.09 | 75.9 | 62.9 | — | — | 51.3 | 79.1 | — | — | — | — | — | — | — | — | — | — | — | 79.8 | — | — | — | |
| IRRRreranking=true2021.06 | 75.6 | 62.1 | — | — | 54.5 | 81.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HopRetriever2020.09 | 75.2 | 62.1 | — | — | 52.5 | 78.9 | — | — | — | — | — | — | — | — | — | — | — | 82.5 | — | — | — | |
| HopRetrieverRetriever=MDR2022.05 | 75.2 | 62.1 | — | — | 52.5 | 78.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MDR-ElectraTop-K paths=Top-50 paths2022.05 | 74.3 | 61.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Recurrent Retriever2020.09 | 73.3 | 60.5 | — | — | 49.3 | 76.1 | — | — | — | — | — | — | — | — | — | — | — | 75.7 | — | — | — | |
| Graph Recurrent RetrieverRetriever=MDR, variant=wwm2022.05 | 73.3 | 60.5 | — | — | 49.3 | 76.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MDR-FiDTop-K paths=Top-50 paths2022.05 | 73.1 | 61.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PATHFID+Top-K paths=Top-25 paths2022.05 | 72.4 | 59.8 | — | — | 52.8 | 76.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| DOCHOPPERreranking=false2021.06 | 69.5 | 57.4 | — | — | 45.1 | 73.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| PATHFIDTop-K paths=Top-25 paths2022.05 | 67.9 | 55.8 | — | — | 49 | 74.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FiD*Top-K paths=Top-25 paths, implementation=re-implemented by current paper2022.05 | 66 | 54 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Graph Recurrent RetrieverRetriever=MDR, variant=base2022.05 | 65.8 | 52.7 | — | — | 47.9 | 75 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| IRRRreranking=false2021.06 | 64.8 | 53.6 | — | — | 44.7 | 73.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Transformer-XH2020.09 | 62.4 | 50.2 | — | — | 42.2 | 71.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Transformer-XHRetriever=MDR2022.05 | 62.4 | 50.2 | — | — | 42.2 | 71.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Semantic Retrieval MRS2020.09 | 58.8 | 46.5 | — | — | 39.9 | 71.5 | — | — | — | — | — | — | — | — | — | — | — | 63.9 | — | — | — | |
| Semantic RetrievalRetriever=MDR2022.05 | 58.8 | 46.5 | — | — | 39.9 | 71.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GoldEn RetrieverRetriever=MDR2022.05 | 49.8 | — | — | — | — | 64.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GoldEn Retriever2020.09 | 49.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CogQA2019.05 | 49.4 | 37.6 | 52.2 | 49.9 | 23.1 | 58.5 | 64.3 | 59.7 | 12.2 | 35.3 | 40.3 | 36.5 | — | — | — | — | — | — | — | — | — | |
| Cognitive Graph2020.09 | 49.4 | 37.6 | — | — | 23.1 | 58.5 | — | — | — | — | — | — | — | — | — | — | — | 57.8 | — | — | — | |
| DrKIT2020.09 | 46.6 | 35.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CogQAvariant=onlyR2019.05 | 46.2 | 34.6 | 48.8 | 46.7 | 14.7 | 48.2 | 56.4 | 47.7 | 8.3 | 29.9 | 36.2 | 30.1 | — | — | — | — | — | — | — | — | — | |
| CogQAvariant=sys12019.05 | 45 | 33.6 | 47.6 | 45.4 | 23.7 | 58.3 | 67.3 | 56.2 | 12.3 | 32.5 | 39 | 31.8 | — | — | — | — | — | — | — | — | — | |
| DecompRC2020.09 | 43.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MUPPETSetting=full wiki, Representation level=paragraph-level2019.06 | 40.42 | 31.07 | — | — | 17 | 47.71 | — | — | 11.76 | 27.62 | — | — | — | — | — | — | — | — | — | — | — | |
| CogQAvariant=onlyQ2019.05 | 40.4 | 30.7 | 42.9 | 40.7 | 23.4 | 49.9 | 56.5 | 48.5 | 12.4 | 30.1 | 35.2 | 29.9 | — | — | — | — | — | — | — | — | — | |
| MUPPET2020.09 | 40.4 | 31.1 | — | — | 17 | 47.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MUPPETSetting=full wiki, Representation level=sentence-level2019.06 | 39.43 | 30.2 | — | — | 16.57 | 46.13 | — | — | 11.38 | 26.55 | — | — | — | — | — | — | — | — | — | — | — | |
| TF-IDF + ReaderSetting=full wiki2019.06 | 36.58 | 27.55 | — | — | 10.75 | 42.45 | — | — | 7 | 21.47 | — | — | — | — | — | — | — | — | — | — | — | |
| BaselineSetting=full wiki2019.06 | 34.36 | 24.68 | — | — | 5.28 | 40.98 | — | — | 2.54 | 17.73 | — | — | — | — | — | — | — | — | — | — | — | |
| Yang et al. (2018)variant=IR2019.05 | 34 | 24.6 | 35.7 | 34.8 | 10.9 | 49.3 | 52.5 | 52.1 | 5.2 | 21.1 | 22.7 | 23.2 | — | — | — | — | — | — | — | — | — | |
| Yang et al. (2018)2019.05 | 32.9 | 23.9 | 34.9 | 33.9 | 5.1 | 40.9 | 47.2 | 40.8 | 2.5 | 17.2 | 20.4 | 17.8 | — | — | — | — | — | — | — | — | — | |
| BERT2019.05 | 31.6 | 22.7 | 33.4 | 31.9 | 6.5 | 42.4 | 54.6 | 38.7 | 3.1 | 17.8 | 24.3 | 16.2 | — | — | — | — | — | — | — | — | — | |
| BERTTraining=Standard2019.06 | — | — | — | — | — | — | — | — | — | — | — | — | 38.4 | 34.77 | 52.85 | 46.14 | 31.74 | — | — | — | — | |
| BERTTraining=1hop train (single-hop QA data only)2019.06 | — | — | — | — | — | — | — | — | — | — | — | — | 29.97 | 32.15 | 21.29 | 47.14 | 15.18 | — | — | — | — | |
| BiDAFTraining=Standard2019.06 | — | — | — | — | — | — | — | — | — | — | — | — | 34.36 | 30.42 | 50.7 | — | — | — | — | — | — | |
| DECOMPRCTraining=Standard2019.06 | — | — | — | — | — | — | — | — | — | — | — | — | 43.26 | 40.3 | 55.04 | 52.11 | 35.64 | — | — | — | — | |
| DECOMPRCTraining=1hop train (single-hop QA data only)2019.06 | — | — | — | — | — | — | — | — | — | — | — | — | 39.17 | 35.3 | 54.57 | 50.03 | 29.83 | — | — | — | — | |
| FRUGALRAG 7BRetriever=ColBERTv2, Answer Generation Model=Qwen2.5-7B-Instruct2025.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 68.47 | 82.8 | 2.05 | |
| FRUGALRAG-Explore 7BRetriever=ColBERTv2, Answer Generation Model=Qwen2.5-7B-Instruct2025.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 67.7 | 86.4 | 5.99 | |
| ReAct + DsPyRetriever=ColBERTv2, Answer Generation Model=Qwen2.5-7B-Instruct2025.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 64.2 | 79.6 | 2.76 | |
| Zero-ShotRetriever=ColBERTv2, Answer Generation Model=Qwen2.5-7B-Instruct2025.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 28.1 | — | — | |
| Zero-Shot CoTRetriever=ColBERTv2, Answer Generation Model=Qwen2.5-7B-Instruct2025.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 29.1 | — | — | |
| Zero-Shot RAGRetriever=ColBERTv2, Answer Generation Model=Qwen2.5-7B-Instruct2025.07 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 51 | 59.05 | 1 |