Open-domain Question Answering on TriviaQA open (test)
73.3EMFiD-large++
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FiD-large++Model Category=Generative, #θ=848M2021.09 | 73.3 | — | |
| UnitedQA (Ens. E+G+G)Model Category=Generative, #θ=1.87B2021.09 | 70.5 | — | |
| R2-D2 (21M)#θ=1.29B2021.09 | 69.9 | — | |
| UnitedQA-EModel Category=Extractive, #θ=440M2021.09 | 68.9 | — | |
| UnitedQA-GModel Category=Generative, #θ=880M2021.09 | 68.6 | — | |
| Fusion-in-DecoderModel scale=large2020.07 | 67.6 | — | |
| FIDReader Type=Generative2020.09 | 67.6 | — | |
| FiD-largeModel Category=Generative, #θ=848M2021.09 | 67.6 | — | |
| R1-D1 (Generative)Model Category=Generative, #θ=848M2021.09 | 65.4 | — | |
| Fusion-in-DecoderModel scale=base2020.07 | 65 | — | |
| FiD-baseModel Category=Generative, #θ=333M2021.09 | 65 | — | |
| R1-D1 (Extractive)Model Category=Extractive, #θ=445M2021.09 | 65 | — | |
| Multi-Passage BERTBackbone=BERT-Large2019.08 | 63.7 | 69.2 | |
| HAS-QA2019.08 | 63.6 | 68.9 | |
| ColBERTModel Category=Extractive, #θ=440M2021.09 | 63.2 | — | |
| GARReader Type=Extractive2020.09 | 62.7 | — | |
| GARReader Type=Generative2020.09 | 62.2 | — | |
| Multi-Passage BERTBackbone=BERT-Base2019.08 | 62 | 67.5 | |
| T5-1.1+SSMModel Category=Generative, #θ=11B2021.09 | 61.6 | — | |
| BERT-RC2019.08 | 61 | 66.9 | |
| T5Reader Type=Generative2020.09 | 60.5 | — | |
| BM25 (ours)Reader Type=Extractive2020.09 | 60.1 | — | |
| BM25+BERTModel Category=Extractive, #θ=110M2021.09 | 60.1 | — | |
| BM25 (ours)Reader Type=Generative2020.09 | 58.6 | — | |
| BM25+SSGModel Category=Generative, #θ=406M2021.09 | 58.6 | — | |
| DPR2020.07 | 57.9 | — | |
| DPRReader Type=Extractive2020.09 | 57.9 | — | |
| ANCE + ReaderReader=DPR Reader2020.07 | 57.5 | — | |
| DPR + BM25retrieval=DPR + BM252020.07 | 57 | — | |
| RDRModel Category=Extractive, #θ=110M2021.09 | 57 | — | |
| DPR2020.07 | 56.8 | — | |
| DPRModel Category=Extractive, #θ=220M2021.09 | 56.8 | — | |
| RAGModel Category=Generative, #θ=516M2021.09 | 56.8 | — | |
| RAG-Sequence2020.07 | 56.1 | — | |
| RAG2020.07 | 56.1 | — | |
| RAGReader Type=Generative2020.09 | 56.1 | — | |
| Graph RetrieverReader Type=Extractive2020.09 | 56 | — | |
| Graph RetrieverModel Category=Extractive, #θ=110M2021.09 | 56 | — | |
| Graph Retriever2020.07 | 55.8 | — | |
| RAG-Token2020.07 | 55.2 | — | |
| Discrete Hard EM2019.09 | 50.9 | — | |
| Hard EM2020.07 | 50.9 | — | |
| Hard EMReader Type=Extractive2020.09 | 50.9 | — | |
| Hard EMModel Category=Extractive, #θ=110M2021.09 | 50.9 | — | |
| Full Re-Rankercombination=three different re-rankers2017.11 | 50.6 | 57.3 | |
| Coverage-Based Re-Ranker2017.11 | 50 | 57 | |
| Strength-Based Re-Rankervariant=Probability2017.11 | 49.2 | 55.1 | |
| OpenQA2019.08 | 48.7 | 56.3 | |
| First Only2019.09 | 48.1 | — | |
| MML2019.09 | 47.4 | — | |
| R³2017.11 | 47.3 | 53.7 | |
| R³2019.08 | 47.3 | 53.7 | |
| Lee et al. (2019)2019.09 | 47.1 | — | |
| Strength-Based Re-Rankervariant=Counting2017.11 | 46.1 | 55.8 | |
| ORQA2020.07 | 45.1 | — | |
| ORQAReader Type=Extractive2020.09 | 45 | — | |
| ORQAModel Category=Extractive, #θ=220M2021.09 | 45 | — | |
| Baseline Re-Ranker (BM25)2017.11 | 44.6 | 55.7 | |
| DrQA2019.08 | 32.3 | 38.3 | |
| Full-Training2026.05 | — | 12.6 | |
| Global Subset2026.05 | — | 13.7 | |
| Layer-Wise Subset2026.05 | — | 14.6 |