Open-set knowledge retrieval on T-REx (UHN)
0.316Macro EMBM25 + GPT-3 175B
Evaluation results
| Method | Links | |
|---|---|---|
| BM25 + GPT-3 175B#Params=500x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.316 | |
| NPM#Params=1.0x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.29 | |
| GPT-3 175B#Params=500x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.241 | |
| BM25 + GPT-3 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.227 | |
| BM25 + T5#Params=2.2x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.203 | |
| BM25 + OPT 13B#Params=37x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.191 | |
| BM25 + T5 3B#Params=8.5x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.19 | |
| BM25 + GPT-3 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.153 | |
| BM25 + OPT 6.7B#Params=19x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.143 | |
| BM25 + OPT 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.141 | |
| GPT-3 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.137 | |
| OPT 13B#Params=37x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.127 | |
| OPT 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.099 | |
| T5 3B#Params=8.5x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.082 | |
| OPT 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.067 | |
| T5#Params=2.2x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.055 | |
| GPT-3 6.7B#Params=19x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.05 | |
| BM25 + GPT-3 2.7B#Params=7.6x, C (Text Corpus)=true, Zero-shot=true2022.12 | 0.034 | |
| GPT-3 2.7B#Params=7.6x, C (Text Corpus)=false, Zero-shot=true2022.12 | 0.026 |