Document Retrieval on NQ 100K
27.5Hits@1DSI (Naive String Docid)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DSI (Naive String Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 27.5 | 62.4 | |
| DSI (Semantic String Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 26.9 | 59.5 | |
| DSI (Atomic Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 25.3 | 67.9 | |
| Dual EncoderModel=T5, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 24.1 | 64.5 | |
| DSI (Naive String Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 24 | 55.1 | |
| DSI (Atomic Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 23.8 | 58.6 | |
| Dual EncoderModel=T5, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 23.3 | 63.2 | |
| DSI Atomic DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 23 | 57.3 | |
| DSI (Semantic String Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 22.4 | 52.2 | |
| Dual EncoderModel=T5, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 22.3 | 60.5 | |
| DSI (Naive String Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 21.2 | 50.7 | |
| BM25Evaluation Protocol=Zero-Shot2022.02 | 20.9 | 46.4 | |
| BM25Evaluation Protocol=Supervised Finetuning2022.02 | 20.9 | 46.4 | |
| DSI (Semantic String Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 20.4 | 50.2 | |
| DSI (Atomic Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 19 | 55.3 | |
| DSI (Semantic String Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 19 | 44.9 | |
| Dual EncoderModel=T5, Size=Base, Params=220M, Evaluation Protocol=Supervised Finetuning2022.02 | 18.7 | 55.2 | |
| DSI (Naive String Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 18.7 | 44.6 | |
| SentenceT5 Dual EncoderModel=SentenceT5, Size=Large, Evaluation Protocol=Zero-Shot2022.02 | 17.4 | 50.8 | |
| DSI Naive String DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 17.4 | 41.5 | |
| DSI (Atomic Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 17.1 | 52.3 | |
| DSI Semantic String DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 11.4 | 26.6 | |
| T5 Dual EncoderModel=T5, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 1.9 | 8 |