Open-Domain Question Answering on NaturalQuestions (test)
54.4Top-1 EMReader-to-Retriever Distillation
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Reader-to-Retriever DistillationRetriever Initialization=DPR, Reader Backbone=T5 large2020.12 | 54.4 | — | — | — | — | |
| Fusion-in-DecoderReader Backbone=T5 large2020.12 | 51.4 | — | — | — | — | |
| Reader-to-Retriever DistillationRetriever Initialization=DPR, Reader Backbone=T5 base2020.12 | 50.1 | — | — | — | — | |
| Reader-to-Retriever DistillationRetriever Initialization=BM25, Reader Backbone=T5 base2020.12 | 48.9 | — | — | — | — | |
| ColBERT-QA2020.12 | 48.2 | — | — | — | — | |
| Fusion-in-DecoderReader Backbone=T5 base2020.12 | 48.2 | — | — | — | — | |
| RAG2020.12 | 44.5 | — | — | — | — | |
| DPR2020.12 | 41.5 | — | — | — | — | |
| Reader-to-Retriever DistillationRetriever Initialization=BERT, Reader Backbone=T5 base2020.12 | 40 | — | — | — | — | |
| APE+FiD-baseModel variant=base, Adaptive Passage Encoder=true2021.07 | — | 40.3 | 43.7 | 46 | 48.2 | |
| APE+FiD-largeModel variant=large, Adaptive Passage Encoder=true2021.07 | — | 43.4 | 46.6 | 49.1 | 51.4 | |
| DPR (Karpukhin et al., 2020)Adaptive Passage Encoder=false2021.07 | — | — | 40.8 | — | 41.5 | |
| DPR (our implementation)Adaptive Passage Encoder=false2021.07 | — | 38.4 | 40.2 | 40.2 | 40.2 | |
| FiD-baseModel variant=base, Adaptive Passage Encoder=false2021.07 | — | 39.5 | 42.9 | 45.3 | 48.2 | |
| FiD-largeModel variant=large, Adaptive Passage Encoder=false2021.07 | — | 42.5 | 45.8 | 48.3 | 51.4 | |
| RAGAdaptive Passage Encoder=false2021.07 | — | 43.5 | 44.1 | 44.1 | 44.5 | |
| SkylineBuilderAdaptive Passage Encoder=false2021.07 | — | 34.4 | 34.2 | 34.2 | — |