Open-domain Question Answering on TriviaQA (Macro-averaged EM)
49Macro EMGPT-3 175B
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-3 175B#Params=500x, C (Text Corpus)=false, Zero-shot=true2022.12 | 49 | |
| BM25 + GPT-3 175B#Params=500x, C (Text Corpus)=true, Zero-shot=true2022.12 | 44.9 | |
| BM25 + T5 3B#Params=8.5x, C (Text Corpus)=true, Zero-shot=true2022.12 | 39.6 | |
| BM25 + OPT 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.12 | 34 | |
| BM25 + GPT-3 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.12 | 32.4 | |
| NPM#Params=1.0x, C (Text Corpus)=true, Zero-shot=true2022.12 | 32.2 | |
| BM25 + T5#Params=2.2x, C (Text Corpus)=true, Zero-shot=true2022.12 | 31.4 | |
| BM25 + OPT 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.12 | 29.9 | |
| BM25 + OPT 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.12 | 28.5 | |
| GPT-3 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.12 | 25.5 | |
| OPT 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.12 | 22.5 | |
| BM25 + GPT-3 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.12 | 21.1 | |
| OPT 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.12 | 20.9 | |
| BM25 + GPT-3 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.12 | 14.5 | |
| GPT-3 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.12 | 12.4 | |
| OPT 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.12 | 9.9 | |
| T5 3B#Params=8.5x, C (Text Corpus)=false, Zero-shot=true2022.12 | 9 | |
| GPT-3 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.12 | 5.2 | |
| T5#Params=2.2x, C (Text Corpus)=false, Zero-shot=true2022.12 | 4.2 |