Document Retrieval on NQ10K
48.5Hits@1DSI (Semantic String Docid)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| DSI (Semantic String Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 48.5 | 72.1 | |
| DSI (Naive String Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 46.7 | 77.9 | |
| DSI (Naive String Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 44.7 | 66.4 | |
| DSI Semantic String DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 43.9 | 68.8 | |
| DSI Naive String DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 43.4 | 67.4 | |
| DSI (Semantic String Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 41.9 | 67.1 | |
| DSI (Atomic Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 40.1 | 76.9 | |
| DSI (Atomic Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 39.4 | 77 | |
| DSI (Semantic String Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 37.5 | 65.1 | |
| DSI (Naive String Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 34.7 | 60.5 | |
| DSI (Semantic String Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 33.9 | 57.3 | |
| DSI (Atomic Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 31.3 | 59.4 | |
| DSI (Naive String Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 28.1 | 48 | |
| DSI Atomic DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 25.7 | 60.1 | |
| Dual EncoderModel=T5, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 22.1 | 61.6 | |
| Dual EncoderModel=T5, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 20.8 | 59.6 | |
| Dual EncoderModel=T5, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 18.8 | 55.7 | |
| SentenceT5 Dual EncoderModel=SentenceT5, Size=Large, Evaluation Protocol=Zero-Shot2022.02 | 17.6 | 50.7 | |
| Dual EncoderModel=T5, Size=Base, Params=220M, Evaluation Protocol=Supervised Finetuning2022.02 | 16.2 | 48.6 | |
| DSI (Atomic Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 13 | 38.4 | |
| BM25Evaluation Protocol=Zero-Shot2022.02 | 12.4 | 33.5 | |
| BM25Evaluation Protocol=Supervised Finetuning2022.02 | 12.4 | 33.5 | |
| T5 Dual EncoderModel=T5, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 0.3 | 1.3 |