List your company
ResearchFormerly WizWand

Open-domain Question Answering on TriviaQA (Macro-averaged EM)

49Macro EMGPT-3 175B
2.40814.50426.638.696Dec 2, 2022
Updated 8mo ago

Evaluation results

MethodLinks
GPT-3 175B#Params=500x, C (Text Corpus)=false, Zero-shot=true2022.1249
BM25 + GPT-3 175B#Params=500x, C (Text Corpus)=true, Zero-shot=true2022.1244.9
BM25 + T5 3B#Params=8.5x, C (Text Corpus)=true, Zero-shot=true2022.1239.6
BM25 + OPT 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.1234
BM25 + GPT-3 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.1232.4
NPM#Params=1.0x, C (Text Corpus)=true, Zero-shot=true2022.1232.2
BM25 + T5#Params=2.2x, C (Text Corpus)=true, Zero-shot=true2022.1231.4
BM25 + OPT 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.1229.9
BM25 + OPT 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.1228.5
GPT-3 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.1225.5
OPT 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.1222.5
BM25 + GPT-3 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.1221.1
OPT 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.1220.9
BM25 + GPT-3 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.1214.5
GPT-3 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.1212.4
OPT 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.129.9
T5 3B#Params=8.5x, C (Text Corpus)=false, Zero-shot=true2022.129
GPT-3 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.125.2
T5#Params=2.2x, C (Text Corpus)=false, Zero-shot=true2022.124.2