Document Retrieval on NQ 320k (test)
63.4Hits@1GENRE*-CONTRA
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GENRE*-CONTRABackbone=T5-large, Decoding=Np Decoding (CONTRA)2022.10 | 63.4 | 81.1 | |
| GENRE*-BASEBackbone=T5-large, Decoding=Np Decoding (BASE)2022.10 | 62.2 | 78.8 | |
| DSINaive-CONTRADecoding=Np Decoding (CONTRA), Identifier=Naively structured identifiers2022.10 | 60.4 | 75.8 | |
| DSINaive-BASEDecoding=Np Decoding (BASE), Identifier=Naively structured identifiers2022.10 | 58.7 | 73.1 | |
| GENRE*Backbone=T5-large, Decoding=Vanilla2022.10 | 53.7 | 64.7 | |
| DSIsemanticIdentifier=Semantic identifiers2022.10 | 35.6 | 62.6 | |
| TCT-ColBERTModel Size=Base2023.05 | 24.11 | 71.97 | |
| DSI-QG-Multi (sem)Model Size=Base, Task=Retrv, semantic IDs=true2023.05 | 23.92 | 71.35 | |
| DSI-QG-Multi (sem)Model Size=Base, Task=Index, semantic IDs=true2023.05 | 23.9 | 72.02 | |
| DSI-QG-MultiModel Size=Large, Task=Retrv2023.05 | 22.85 | 71.1 | |
| Sentence-T52022.10 | 22.4 | 63.3 | |
| DSI-QGModel Size=Large, Task=Index2023.05 | 22.23 | 51.89 | |
| DSI-QG-MultiModel Size=Large, Task=Index2023.05 | 22.06 | 63.04 | |
| DSI-QGModel Size=Base, Task=Index2023.05 | 21.93 | 51.8 | |
| DSI-QG-MultiModel Size=Base, Task=Index2023.05 | 21.9 | 61.38 | |
| DSI-QG-MultiModel Size=Base, Task=Retrv2023.05 | 21.62 | 70.03 | |
| NCI (sem)Model Size=Base, Task=Index, semantic IDs=true2023.05 | 21.2 | 69.99 | |
| SEALModel Size=Base2023.05 | 13.77 | 56.79 | |
| DSINaiveIdentifier=Naively structured identifiers2022.10 | 13.3 | 33.6 | |
| BM252022.10 | 11.6 | 34.4 | |
| BM252023.05 | 8.34 | 33.36 | |
| DSIModel Size=Large, Task=Index2023.05 | 0.57 | 4.93 | |
| DSIModel Size=Base, Task=Index2023.05 | 0.08 | 0.83 |