List your company
ResearchFormerly WizWand

Open-set knowledge retrieval on T-REx (Hard)

32.1Macro EMNPM
2.4610.15517.8525.545Dec 2, 2022
Updated 8mo ago

Evaluation results

MethodLinks
NPM#Params=1.0x, C (Text Corpus)=true, Zero-shot=true2022.1232.1
BM25 + GPT-3 175B#Params=500x, C (Text Corpus)=true, Zero-shot=true2022.1231.3
GPT-3 175B#Params=500x, C (Text Corpus)=false, Zero-shot=true2022.1224.7
BM25 + T5#Params=2.2x, C (Text Corpus)=true, Zero-shot=true2022.1222.4
BM25 + GPT-3 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.1222.4
BM25 + T5 3B#Params=8.5x, C (Text Corpus)=true, Zero-shot=true2022.1221.8
BM25 + OPT 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.1219.3
GPT-3 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.1215.5
BM25 + GPT-3 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.1215.1
BM25 + OPT 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.1214.9
BM25 + OPT 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.1213.8
OPT 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.1212.7
T5 3B#Params=8.5x, C (Text Corpus)=false, Zero-shot=true2022.1211.5
T5#Params=2.2x, C (Text Corpus)=false, Zero-shot=true2022.1210.7
OPT 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.1210.7
OPT 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.128.3
GPT-3 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.126.7
GPT-3 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.123.8
BM25 + GPT-3 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.123.6