Passage Retrieval on WebQuestions (WQ) (test)
85.4Top-20 AccuracyGAR best query
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GAR best queryTraining Setting=Transfer from NQ2023.05 | 85.4 | 90.3 | 78.9 | |
| GAR best queryTraining Setting=Transfer from TriviaQA2023.05 | 84.6 | 89.3 | 78.4 | |
| ICT+WLP+BFS+Evaluation protocol=fine-tuning2022.03 | 76.7 | 84.4 | 64.1 | |
| HLPEvaluation protocol=fine-tuning2022.03 | 76.5 | 84.5 | 65.5 | |
| ICTEvaluation protocol=fine-tuning2022.03 | 75.5 | 83.4 | 63.7 | |
| WLPEvaluation protocol=fine-tuning2022.03 | 75.2 | 83.9 | 64.5 | |
| BFS+Evaluation protocol=fine-tuning2022.03 | 75.1 | 83.5 | 63 | |
| DPRTraining=Multi, Retriever=DPR2020.04 | 75 | 82.9 | — | |
| DPR-supervised + C-MORESettings=Supervised, Training Data=Supervised Data + Wikipedia2022.03 | 75 | 83.2 | — | |
| BM25 + DPRTraining=Multi, Retriever=BM25 + DPR2020.04 | 74.7 | 82.3 | — | |
| No Pre-trainEvaluation protocol=fine-tuning2022.03 | 74.5 | 81.7 | 61.6 | |
| DPR-TQA + C-MORESettings=Domain Adaptation, Source Training Data=TriviaQA + Wikipedia2022.03 | 74.3 | 83.2 | — | |
| DPR+Training Setting=In-Domain Supervised2023.05 | 74.3 | 82.2 | 62.8 | |
| DPRTraining=Single, Retriever=DPR2020.04 | 73.2 | 81.4 | — | |
| DPR-supervisedSettings=Supervised, Training Data=Supervised Data2022.03 | 73.2 | 81.4 | — | |
| DPR-TQASettings=Domain Adaptation, Source Training Data=TriviaQA2022.03 | 71.5 | 81 | — | |
| DPR+Training Setting=Transfer from TriviaQA2023.05 | 71.4 | 81.2 | 56.8 | |
| DPR-NQ + C-MORESettings=Domain Adaptation, Source Training Data=NaturalQuestion + Wikipedia2022.03 | 71.2 | 81.3 | — | |
| BM25 + DPRTraining=Single, Retriever=BM25 + DPR2020.04 | 71 | 80.5 | — | |
| EAR-RDTraining Setting=Transfer from NQ2023.05 | 70.8 | 81.3 | 59.5 | |
| EAR-RITraining Setting=Transfer from NQ2023.05 | 69.6 | 81.3 | 53.7 | |
| DPR-NQSettings=Domain Adaptation, Source Training Data=NaturalQuestion2022.03 | 69 | 78.8 | — | |
| DPR+Training Setting=Transfer from NQ2023.05 | 68.8 | 78.3 | 52.7 | |
| EAR-RDTraining Setting=Transfer from TriviaQA2023.05 | 68 | 79.7 | 54.5 | |
| EAR-RITraining Setting=Transfer from TriviaQA2023.05 | 67.1 | 79.6 | 49.6 | |
| HLPEvaluation protocol=zero-shot2022.03 | 66.9 | 80.8 | 49.3 | |
| GARTraining Setting=Transfer from NQ2023.05 | 66 | 79 | 50 | |
| BM25Evaluation protocol=zero-shot2022.03 | 62.8 | 76.8 | 42.6 | |
| C-MORESettings=Unsupervised, Training Data=Wikipedia2022.03 | 62.7 | 78.5 | — | |
| BM25Training Setting=N/A2023.05 | 62.4 | 75.5 | 41.8 | |
| GARTraining Setting=Transfer from TriviaQA2023.05 | 61.8 | 76.7 | 45.5 | |
| BM25Training=None, Retriever=BM252020.04 | 55 | 71.1 | — | |
| BM25Settings=Unsupervised, Training Data=None2022.03 | 55 | 71.1 | — | |
| WLPEvaluation protocol=zero-shot2022.03 | 49 | 68.1 | 26.9 | |
| BFS+Evaluation protocol=zero-shot2022.03 | 48 | 67.7 | 28.5 | |
| ICT+WLP+BFS+Evaluation protocol=zero-shot2022.03 | 47.8 | 67.5 | 28.4 | |
| ICTEvaluation protocol=zero-shot2022.03 | 36.2 | 56 | 19.9 |