Retrieval on HotpotQA full wiki (dev)
88.17PEMAISO
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AISOStrategy=(fs || fl || fi)Pi, Model scale=large2021.09 | 88.17 | — | — | — | — | — | — | — | — | — | — | — | — | 35.7 | |
| HopRetriever-plusStrategy=fso o fl-1, Model scale=large2021.09 | 86.94 | — | — | — | — | — | — | — | — | — | — | — | — | 500 | |
| BaleenModel scale=large2021.09 | 86.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| TPRRStrategy=fso o fl-1, Model scale=large2021.09 | 86.19 | — | — | — | — | — | — | — | — | — | — | — | — | 500 | |
| AISOStrategy=(fs || fl || fi)Pi, Model scale=base2021.09 | 85.69 | — | — | — | — | — | — | — | — | — | — | — | — | 36.7 | |
| IRRRModel scale=large2021.09 | 84.1 | — | — | — | — | — | — | — | — | — | — | — | — | 150 | |
| HopRetrieverStrategy=fso o fl-12021.09 | 82.54 | — | — | — | — | — | — | — | — | — | — | — | — | 500 | |
| MDR + RerankerStrategy=fd o fd, Model scale=large2021.09 | 81.2 | — | — | — | — | — | — | — | — | — | — | — | — | 200 | |
| DDRQAModel scale=large2021.09 | 79.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| GRRModel scale=large2021.09 | 75.7 | — | — | — | — | — | — | — | — | — | — | — | — | 500 | |
| Semantic RetrievalStrategy=fa2021.09 | 69.35 | — | — | — | — | — | — | — | — | — | — | — | — | 39.4 | |
| MDRStrategy=fd o fd2021.09 | 64.52 | — | — | — | — | — | — | — | — | — | — | — | — | 2 | |
| CogQA2021.09 | 57.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Golden RetrieverStrategy=fs o fs2021.09 | 47.77 | — | — | — | — | — | — | — | — | — | — | — | — | 10 | |
| DrKitStrategy=(fs || fa)2021.09 | 38.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Entity Centric IRStrategy=fs, fi2021.09 | 34.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| BM25 + RerankerStrategy=fs2021.09 | 29.6 | — | — | — | — | — | — | — | — | — | — | — | — | 20 | |
| DPRStrategy=fa2021.09 | 14.18 | — | — | — | — | — | — | — | — | — | — | — | — | 2 | |
| BM252021.09 | 11.11 | — | — | — | — | — | — | — | — | — | — | — | — | 2 | |
| Cognitive Graph2019.11 | — | 76 | 87.6 | 57.8 | 37.6 | — | — | — | — | — | — | — | — | — | |
| Cognitive Graph QAdocument sequence generation=single sequence2020.12 | — | 72.21 | — | — | — | 70.25 | — | — | 57.8 | — | — | — | — | — | |
| DPRArchitecture=Bi-Encoder2022.10 | — | — | — | — | — | — | — | — | — | — | — | 25.2 | 45.4 | — | |
| Entity-centric IR2019.11 | — | 63.4 | 87.3 | 34.9 | 42 | — | — | — | — | — | — | — | — | — | |
| GENRE*Architecture=Generative2022.10 | — | — | — | — | — | — | — | — | — | — | — | 56.1 | 58.4 | — | |
| GENRE*-BASEArchitecture=Generative, Decoding=Nonparametric Decoding2022.10 | — | — | — | — | — | — | — | — | — | — | — | 61.9 | 65.3 | — | |
| GENRE*-CONTRAArchitecture=Generative, Decoding=Nonparametric Decoding2022.10 | — | — | — | — | — | — | — | — | — | — | — | 63.1 | 66.8 | — | |
| GMRArchitecture=Generative2022.10 | — | — | — | — | — | — | — | — | — | — | — | 57.7 | 58.8 | — | |
| HopRetrieverbackbone=BERT-base, beam size=82020.12 | — | 86.89 | — | — | — | 88.41 | 92.78 | 93.2 | 82.54 | 88.6 | 89.09 | — | — | — | |
| HopRetriever-plusretrieval backbone=ELECTRA-base, neural ranker=BERT-base, answer extraction backbone=ELECTRA-large2020.12 | — | 89.25 | — | — | — | 91.29 | 95.44 | 95.7 | 86.94 | 93.25 | 93.72 | — | — | — | |
| MDRArchitecture=Bi-Encoder2022.10 | — | — | — | — | — | — | — | — | — | — | — | 65.9 | 77.5 | — | |
| MDR-Architecture=Bi-Encoder, Configuration=without linked negatives, memory bank, and shared encoder2022.10 | — | — | — | — | — | — | — | — | — | — | — | 59.9 | 70.6 | — | |
| PathRetrieverbackbone=BERT-base, beam size=82020.12 | — | 80.96 | — | — | — | 82.05 | 88.69 | 89.48 | 73.91 | 86.12 | 87.39 | — | — | — | |
| Re-rank2019.11 | — | 55.1 | 85.9 | 29.6 | 35.7 | — | — | — | — | — | — | — | — | — | |
| Re-rank 2hop2019.11 | — | 56 | 70.1 | 26.1 | 38.8 | — | — | — | — | — | — | — | — | — | |
| Recurrent Graph-based RetrievalNumber of reasoning paths (F)=202019.11 | — | 87 | 93.3 | 72.7 | 56.8 | — | — | — | — | — | — | — | — | — | |
| Semantic Retrieval2019.11 | — | 77.9 | 93.2 | 63.9 | 46.5 | — | — | — | — | — | — | — | — | — | |
| Semantic Retrievaldocument assembly=top-2/10/16 documents mapped to top-1/5/8 sequences2020.12 | — | 77.84 | — | — | — | 81.68 | 88.35 | 88.5 | 69.35 | 81.73 | 82.07 | — | — | — | |
| TF-IDF2019.11 | — | 39.7 | 66.9 | 10 | 18.2 | — | — | — | — | — | — | — | — | — |