Passage retrieval on TriviaQA (test)
90.1Top-100 AccGAR best query
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GAR best queryRetrieval Paradigm=Lexical Retrieval2023.05 | 90.1 | 88.1 | 85 | — | — | |
| Reader-to-Retriever Distillationinitial passages=DPR2020.12 | 87.7 | 83.6 | — | — | — | |
| FiD-KDTraining Supervision=Supervised2021.12 | 87.7 | 84.6 | 77 | — | — | |
| EAR-RD + DPRRetrieval Paradigm=Fusion (Dense + Lexical) Retrieval2023.05 | 87.3 | 83.7 | 79 | — | — | |
| XTRxxlzero-shot=true2023.04 | 87.1 | 83.3 | — | — | — | |
| EAR-RI + DPRRetrieval Paradigm=Fusion (Dense + Lexical) Retrieval2023.05 | 87 | 83 | 76.4 | — | — | |
| HLPEvaluation protocol=fine-tuning2022.03 | 86.9 | 82.4 | 75.3 | — | — | |
| GAR+fusion=GAR + DPR2020.09 | 86.6 | 82.1 | 76 | — | — | |
| GTRxxlzero-shot=true2023.04 | 86.6 | 81.7 | — | — | — | |
| ICT+WLP+BFS+Evaluation protocol=fine-tuning2022.03 | 86.5 | 82.2 | 74.6 | — | — | |
| Reader-to-Retriever Distillationinitial passages=BM252020.12 | 86.5 | 81.4 | — | — | — | |
| DPRmulti + BM25sparse component=true, in-domain=true2023.04 | 86.5 | 82.6 | — | — | — | |
| EAR-RDRetrieval Paradigm=Lexical Retrieval2023.05 | 86.4 | 82.1 | 77.6 | — | — | |
| Liu et al. (2022) + DPRRetrieval Paradigm=Fusion (Dense + Lexical) Retrieval2023.05 | 86.4 | 82.5 | 76.1 | — | — | |
| GAR + DPRRetrieval Paradigm=Fusion (Dense + Lexical) Retrieval2023.05 | 86.3 | 82.2 | 75.7 | — | — | |
| WLPEvaluation protocol=fine-tuning2022.03 | 86.1 | 81.5 | 73.1 | — | — | |
| TOURhardBackbone=DensePhrases_multi2022.05 | 86.1 | 83.2 | — | — | — | |
| TOURsoftBackbone=DensePhrases_multi2022.05 | 86.1 | 83.2 | — | — | — | |
| BFS+Evaluation protocol=fine-tuning2022.03 | 86 | 80.8 | 72.8 | — | — | |
| EAR-RIRetrieval Paradigm=Lexical Retrieval2023.05 | 85.9 | 80.8 | 73.4 | — | — | |
| DensePhrases_multiModel=Baseline retriever2022.05 | 85.8 | 81.6 | — | — | — | |
| Re-rankerBackbone=DensePhrases_multi, Reference=Fajcik et al., 20212022.05 | 85.8 | 83 | — | — | — | |
| Liu et al. (2022)Retrieval Paradigm=Lexical Retrieval2023.05 | 85.8 | 80.1 | 72.3 | — | — | |
| GAR2020.09 | 85.7 | 80.4 | 73.1 | 88.9 | 89.7 | |
| ICTEvaluation protocol=fine-tuning2022.03 | 85.5 | 79.8 | 70.4 | — | — | |
| XTRbasezero-shot=true2023.04 | 85.5 | 80.3 | — | — | — | |
| ANCE2020.12 | 85.3 | 80.3 | — | — | — | |
| GARRetrieval Paradigm=Lexical Retrieval2023.05 | 85.3 | 79.5 | 71.8 | — | — | |
| DCSRTraining Setting=Single2021.10 | 85.2 | 79.7 | — | — | — | |
| DCSRTraining Setting=Multi2021.10 | 85.2 | 79.6 | — | — | — | |
| TOURsoftBackbone=DPR_multi2022.05 | 85.1 | 81.6 | — | — | — | |
| DPRTraining=Single, Retriever=DPR2020.04 | 85 | 79.4 | — | — | — | |
| DPRTraining Setting=Single2021.10 | 85 | 79.4 | — | — | — | |
| No Pre-trainEvaluation protocol=fine-tuning2022.03 | 85 | 79.7 | 71.3 | — | — | |
| DPR2020.12 | 85 | 79.4 | — | — | — | |
| DPRTraining Supervision=Supervised2021.12 | 85 | 79.4 | — | — | — | |
| BM25 + DPRRetrieval Paradigm=Fusion (Dense + Lexical) Retrieval2023.05 | 85 | 79.7 | 71.5 | — | — | |
| TOURhardBackbone=DPR_multi2022.05 | 84.9 | 81.5 | — | — | — | |
| DPR2020.09 | 84.8 | 80.2 | 72.7 | — | — | |
| DPR_multiModel=Baseline retriever2022.05 | 84.8 | 79 | — | — | — | |
| Re-rankerBackbone=DPR_multi, Reference=Fajcik et al., 20212022.05 | 84.8 | 81.6 | — | — | — | |
| DPRRetrieval Paradigm=Dense Retrieval2023.05 | 84.8 | 80.2 | 72.7 | — | — | |
| DPRmultiretrieval pre-training=true, zero-shot=true2023.04 | 84.8 | 78.9 | — | — | — | |
| DPRTraining=Multi, Retriever=DPR2020.04 | 84.7 | 78.8 | — | — | — | |
| DPRTraining Setting=Multi2021.10 | 84.7 | 78.8 | — | — | — | |
| BM25 + DPRTraining=Single, Retriever=BM25 + DPR2020.04 | 84.5 | 79.8 | — | — | — | |
| BM25 + DPRTraining=Multi, Retriever=BM25 + DPR2020.04 | 84.4 | 79.9 | — | — | — | |
| Reader-to-Retriever Distillationinitial passages=BERT2020.12 | 84.4 | 78.4 | — | — | — | |
| ART MS MARCOzero-shot=true2023.04 | 84.1 | 78 | — | — | — | |
| HLPEvaluation protocol=zero-shot2022.03 | 84 | 76.9 | 65.9 | — | — | |
| BM25implementation=re-evaluated by authors2020.09 | 83.9 | 77.3 | 67.7 | 87.9 | 88.9 | |
| BM25Retrieval Paradigm=Lexical Retrieval2023.05 | 83.9 | 77.3 | 67.7 | — | — | |
| BM25 + RM32020.09 | 83.8 | 77.1 | 67 | 87.7 | 88.9 | |
| GTRbasezero-shot=true2023.04 | 83.4 | 76.2 | — | — | — | |
| BM25Evaluation protocol=zero-shot2022.03 | 83.2 | 76.4 | 66.4 | — | — | |
| BM25Training Supervision=Unsupervised2021.12 | 83.2 | 76.4 | — | — | — | |
| ContrieverTraining Supervision=Unsupervised2021.12 | 83.2 | 74.2 | 59.4 | — | — | |
| BM25sparse component=true, zero-shot=true2023.04 | 83.2 | 76.4 | — | — | — | |
| ColBERTzero-shot=true2023.04 | 80.3 | — | — | — | — | |
| MSSEvaluation protocol=zero-shot2022.03 | 79.4 | 68.2 | 53.3 | — | — | |
| Masked salient spansTraining Supervision=Unsupervised2021.12 | 79.4 | 68.2 | 53.3 | — | — | |
| WLPEvaluation protocol=zero-shot2022.03 | 79.1 | 67 | 51.3 | — | — | |
| ICT+WLP+BFS+Evaluation protocol=zero-shot2022.03 | 78.3 | 65.5 | 49.7 | — | — | |
| BM25Training=None, Retriever=BM252020.04 | 76.7 | 66.9 | — | — | — | |
| BFS+Evaluation protocol=zero-shot2022.03 | 74.7 | 61.1 | 43.8 | — | — | |
| Inverse Cloze TaskTraining Supervision=Unsupervised2021.12 | 73.6 | 57.5 | 40.2 | — | — | |
| ICTEvaluation protocol=zero-shot2022.03 | 69.9 | 51.3 | 33.3 | — | — |