Information Retrieval on NQ320k
48.92Hits@1DDRO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DDRODocument Identifier (docid) Strategy=PQ (Product Quantization)2025.04 | 48.92 | 67.31 | 64.1 | 55.51 | |
| DDRORetrieval Type=Generative, Docid Strategy=Product Quantization2025.04 | 48.92 | 67.31 | 64.1 | 55.51 | |
| DDRODocument Identifier (docid) Strategy=TU (Title + URL)2025.04 | 40.86 | 55.98 | 53.12 | 45.99 | |
| DDRORetrieval Type=Generative, Docid Strategy=Title + URL2025.04 | 40.86 | 55.98 | 53.12 | 45.99 | |
| DSI (Semantic String Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 40.4 | 70.3 | — | — | |
| DSI (Semantic String Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 39.1 | 66.8 | — | — | |
| GenRRLDocument Identifier (docid) Strategy=Sum (document summary)2025.04 | 36.32 | 71.49 | 57.42 | 46.31 | |
| ROGER-UltronRetrieval Type=Generative, Docid Strategy=Title + URL2025.04 | 35.9 | 69.86 | 55.59 | 44.92 | |
| GenRRLDocument Identifier (docid) Strategy=TU (Title + URL)2025.04 | 35.79 | 70.96 | 56.49 | 45.73 | |
| DSI (Semantic String Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 35.6 | 62.6 | — | — | |
| UltronRetrieval Type=Generative, Docid Strategy=Title + URL2025.04 | 33.78 | 67.05 | 54.2 | 42.51 | |
| ROGER-NCIRetrieval Type=Generative, Docid Strategy=Semantic ID2025.04 | 33.2 | 69.8 | 56.34 | 43.45 | |
| LTRGRRetrieval Type=Generative, Docid Strategy=Semantic ID2025.04 | 32.8 | 68.74 | 56.2 | 44.8 | |
| NCIRetrieval Type=Generative, Docid Strategy=Semantic ID2025.04 | 32.69 | 69.2 | 55.82 | 42.84 | |
| MINDERRetrieval Type=Generative, Docid Strategy=Semantic ID2025.04 | 31 | 65.79 | 55.5 | 43.5 | |
| DSI-QGRetrieval Type=Generative, Docid Strategy=Semantic ID2025.04 | 30.17 | 66.37 | 53.2 | 38.85 | |
| SEALRetrieval Type=Generative, Docid Strategy=N-grams2025.04 | 29.3 | 68.53 | 54.12 | 40.34 | |
| DSI (Atomic Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 28.1 | 61.9 | — | — | |
| DSIRetrieval Type=Generative, Docid Strategy=Semantic ID2025.04 | 27.42 | 56.58 | 47.26 | 34.31 | |
| DSI (Semantic String Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 27.4 | 56.6 | — | — | |
| GENREReplication=Authors' own replication2022.04 | 26.3 | 71.2 | — | — | |
| SEALComponents=LM+FM, Constraint=Intersection2022.04 | 26.3 | 74.5 | — | — | |
| UltronRetrieval Type=Generative, Docid Strategy=Product Quantization2025.04 | 25.64 | 65.75 | 53.09 | 37.12 | |
| SEALComponents=LM+FM2022.04 | 25.3 | 72 | — | — | |
| DSI Atomic DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 25.1 | 56.6 | — | — | |
| DSI-BARTBackbone=BART2022.04 | 25 | 63.6 | — | — | |
| ANCERetrieval Type=Dense2025.04 | 24.54 | 69.08 | 54.21 | 36.88 | |
| Dual EncoderModel=T5, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 24.3 | 67.3 | — | — | |
| DSI (Atomic Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 24 | 55.1 | — | — | |
| Dual EncoderModel=T5, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 23.9 | 65.8 | — | — | |
| DSI (Naive String Docid)Model=DSI, Size=XXL, Params=11B, Evaluation Protocol=Supervised Finetuning2022.02 | 23.8 | 55.9 | — | — | |
| DPRRetrieval Type=Dense2025.04 | 22.78 | 68.58 | 53.44 | 35.92 | |
| BM25Implementation=pyserini2022.04 | 22.7 | 59 | — | — | |
| SEALComponents=Language Model (LM), n-grams=52022.04 | 22.6 | 68.7 | — | — | |
| RepBERTRetrieval Type=Dense2025.04 | 22.57 | 65.65 | 52.2 | 35.13 | |
| Sentence-T5Retrieval Type=Dense2025.04 | 22.51 | 65.12 | 52 | 34.95 | |
| Dual EncoderModel=T5, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 22.4 | 63.3 | — | — | |
| SEALComponents=Language Model (LM), n-grams=42022.04 | 22.2 | 68.2 | — | — | |
| SEALComponents=Language Model (LM), n-grams=32022.04 | 21.3 | 66.5 | — | — | |
| DSI (Atomic Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 20.7 | 40.9 | — | — | |
| Dual EncoderModel=T5, Size=Base, Params=220M, Evaluation Protocol=Supervised Finetuning2022.02 | 20.5 | 58.3 | — | — | |
| DocT5QueryRetrieval Type=Term-based2025.04 | 19.07 | 55.83 | 43.88 | 29.55 | |
| SentenceT5 Dual EncoderModel=SentenceT5, Size=Large, Evaluation Protocol=Zero-Shot2022.02 | 16.9 | 51 | — | — | |
| DSI (Naive String Docid)Model=DSI, Size=XL, Params=3B, Evaluation Protocol=Supervised Finetuning2022.02 | 16.7 | 58.1 | — | — | |
| BM25Implementation=gensim2022.04 | 15.3 | 44.5 | — | — | |
| BM25Retrieval Type=Term-based2025.04 | 14.06 | 47.93 | 36.91 | 23.6 | |
| DSI Semantic String DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 13.9 | 31.1 | — | — | |
| DSI (Naive String Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 13.3 | 33.6 | — | — | |
| BM25Evaluation Protocol=Zero-Shot2022.02 | 11.6 | 34.4 | — | — | |
| BM25Evaluation Protocol=Supervised Finetuning2022.02 | 11.6 | 34.4 | — | — | |
| DSI (Atomic Docid)Model=DSI, Size=Large, Params=800M, Evaluation Protocol=Supervised Finetuning2022.02 | 11.6 | 37.6 | — | — | |
| DSI Naive String DocidModel=DSI, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 9.2 | 22.6 | — | — | |
| DSI (Naive String Docid)Model=DSI, Size=Base, Params=250M, Evaluation Protocol=Supervised Finetuning2022.02 | 6.7 | 21 | — | — | |
| T5 Dual EncoderModel=T5, Size=XXL, Evaluation Protocol=Zero-Shot2022.02 | 1.1 | 5.9 | — | — |