End-to-end Open-Domain Question Answering on TriviaQA (test)
71.5Exact Match (EM)EAR RD + FiD
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| EAR RD + FiDInput Passages to FiD=1002023.05 | 71.5 | — | — | — | — | |
| EAR RI + FiDInput Passages to FiD=1002023.05 | 71.2 | — | — | — | — | |
| SEAL + FiDInput Passages to FiD=1002023.05 | 70.8 | — | — | — | — | |
| Liu et al. (2022) + FiDInput Passages to FiD=1002023.05 | 70.8 | — | — | — | — | |
| UnitedQAReader Type=Hybrid, Reader Size (M)=2x770+3302021.01 | 70.5 | — | — | — | — | |
| GAR + FiDInput Passages to FiD=1002023.05 | 70 | — | — | — | — | |
| UnitedQA-Elarge++Reader Type=Ensemble, Reader Size (M)=3x3302021.01 | 69.6 | — | — | — | — | |
| UnitedQA-Glarge++Reader Type=Ensemble, Reader Size (M)=3x7702021.01 | 69.2 | — | — | — | — | |
| UnitedQA-ElargeReader Type=Extractive, Reader Size (M)=3302021.01 | 68.9 | — | — | — | — | |
| UnitedQA-GlargeReader Type=Generative, Reader Size (M)=7702021.01 | 68.6 | — | — | — | — | |
| FiD (large)Pretrained model=T5-large, number of parameters=770M2021.06 | 67.6 | — | — | — | — | |
| T5-FIDlargeReader Type=Generative, Reader Size (M)=7702021.01 | 67.6 | — | — | — | — | |
| DPR + FiDInput Passages to FiD=1002023.05 | 67.6 | — | — | — | — | |
| UnitedQA-EbaseReader Type=Extractive, Reader Size (M)=1102021.01 | 66.3 | — | — | — | — | |
| BPRPretrained model=ELECTRA-large, number of parameters=335M, k=10002021.06 | 65.6 | — | — | — | — | |
| FiD (base)Pretrained model=T5-base, number of parameters=220M2021.06 | 65 | — | — | — | — | |
| T5-FIDbaseReader Type=Generative, Reader Size (M)=2202021.01 | 65 | — | — | — | — | |
| EAR RD + FiDInput Passages to FiD=102023.05 | 60 | — | — | — | — | |
| Individual Top-kReader Model Size=large2021.01 | 59.6 | — | — | — | — | |
| BM25+DPRTraining=Multi2020.04 | 57.9 | — | — | — | — | |
| DPRReader Type=Extractive, Reader Size (M)=1102021.01 | 57.9 | — | — | — | — | |
| DPR + ExtractiveInput Passages to FiD=1002023.05 | 57.9 | — | — | — | — | |
| BM25+DPRTraining=Single2020.04 | 57 | — | — | — | — | |
| DPRTraining=Single2020.04 | 56.8 | — | — | — | — | |
| DPRTraining=Multi2020.04 | 56.8 | — | — | — | — | |
| BPRPretrained model=BERT-base, number of parameters=110M, k=10002021.06 | 56.8 | — | — | — | — | |
| DPRReader Model Size=base2021.01 | 56.8 | — | — | — | — | |
| RAGReader Model Size=large2021.01 | 56.8 | — | — | — | — | |
| EAR RI + FiDInput Passages to FiD=102023.05 | 56.7 | — | — | — | — | |
| Individual Top-kReader Model Size=base2021.01 | 56.3 | — | — | — | — | |
| RAGPretrained model=BART-large, number of parameters=406M2021.06 | 56.1 | — | — | — | — | |
| RAGReader Type=Generative, Reader Size (M)=4002021.01 | 56.1 | — | — | — | — | |
| RAGInput Passages to FiD=1002023.05 | 56.1 | — | — | — | — | |
| GraphRetrieverTraining=Single2020.04 | 56 | — | — | — | — | |
| BM25Training=Single2020.04 | 52.4 | — | — | — | — | |
| HardEMTraining=Single2020.04 | 50.9 | — | — | — | — | |
| GAR + FiDInput Passages to FiD=102023.05 | 48.9 | — | — | — | — | |
| BM25+BERTTraining=Single2020.04 | 47.1 | — | — | — | — | |
| ORQATraining=Single2020.04 | 45 | — | — | — | — | |
| ORQAReader Model Size=base2021.01 | 45 | — | — | — | — | |
| APE+FiD-baseModel variant=base, Adaptive Passage Encoder=true2021.07 | — | 55.4 | 59 | 62 | 65 | |
| APE+FiD-largeModel variant=large, Adaptive Passage Encoder=true2021.07 | — | 57.9 | 61.4 | 64.1 | 67.6 | |
| DPR (Karpukhin et al., 2020)Adaptive Passage Encoder=false2021.07 | — | — | — | — | 57.9 | |
| FiD-baseModel variant=base, Adaptive Passage Encoder=false2021.07 | — | 53.9 | 57.9 | 60.7 | 65 | |
| FiD-largeModel variant=large, Adaptive Passage Encoder=false2021.07 | — | 57.2 | 60.6 | 63.7 | 67.6 | |
| RAGAdaptive Passage Encoder=false2021.07 | — | — | — | — | 56.1 |