Open-Domain Question Answering on WebQuestions (WebQ) (dev)
51.5Exact MatchPAQ + FiE
Evaluation results
| Method | Links | |
|---|---|---|
| PAQ + FiEModel Scale=Base2022.11 | 51.5 | |
| PAQ + FiEModel Scale=Large2022.11 | 50.1 | |
| EMDR2top-K=50, # of params=440M, decoding strategy=greedy2021.06 | 49.9 | |
| FiEModel Scale=Large2022.11 | 49.9 | |
| FiD (DPR retriever, T5 reader) (K=50)top-K=50, # of params=440M, decoding strategy=greedy2021.06 | 46 | |
| RAG-Sequencetop-K=50, # of params=626M, pretrained on NQ=true2021.06 | 44.9 | |
| FiEModel Scale=Base2022.11 | 44.3 | |
| FiD (MSS + DPR retriever, T5 reader)top-K=50, # of params=440M, decoding strategy=greedy2021.06 | 43.5 | |
| GPT-3top-K=0, # of params=175B2021.06 | 41.5 | |
| FiD (DPR retriever, T5 reader) (K=1)top-K=1, # of params=440M, decoding strategy=greedy2021.06 | 40.2 | |
| FiD (MSS retriever, MSS reader)top-K=50, # of params=440M, decoding strategy=greedy2021.06 | 39.1 | |
| ORQAtop-K=5, # of params=330M2021.06 | 36.8 | |
| T5-XXLtop-K=0, # of params=11B2021.06 | 35.6 | |
| FiD / T5-base (Ours)top-K=0, # of params=220M, decoding strategy=greedy2021.06 | 31 | |
| T5-largetop-K=0, # of params=770M2021.06 | 29.5 | |
| T5-basetop-K=0, # of params=220M2021.06 | 28.2 | |
| BM25 + BERTtop-K=5, # of params=220M2021.06 | 27.1 |