Retrieval on TREC DL 2019
75.2NDCG@10SPLADE-v3-DistilBERT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SPLADE-v3-DistilBERTRetriever Type=Sparse Retriever2024.11 | 75.2 | — | — | — | — | |
| SPLADE-v3-DistilBERTFLOPS=1.42024.03 | 75.2 | — | — | — | — | |
| BERT RerankerEvaluation Protocol=Rerank2020.07 | 74.2 | — | — | — | — | |
| SPLADE++SDFLOPS=1.42024.03 | 73 | — | — | — | — | |
| DistilSPLADE-maxRetriever Type=Sparse Retriever2024.11 | 72.9 | 86.5 | — | — | — | |
| DistilSPLADE-maxRetrieval Type=Our methods2021.09 | 72.9 | 86.5 | — | — | — | |
| SPLADE-v3FLOPS=1.22024.03 | 72.3 | — | — | — | — | |
| Our ModelRetriever Type=Inference-free Sparse Retriever, Evaluated by Current Authors=true2024.11 | 72.1 | 79.8 | — | — | — | |
| TCT-ColBERTRetriever Type=Dense Retriever2024.11 | 71.9 | 76 | — | — | — | |
| TCT-ColBERTRetrieval Type=Dense retrieval2021.09 | 71.9 | 76 | — | — | — | |
| TAS-BRetriever Type=Dense Retriever2024.11 | 71.7 | 84.3 | — | — | — | |
| TAS-BRetrieval Type=Dense retrieval2021.09 | 71.7 | 84.3 | — | — | — | |
| SPLADE-v3-DocRetriever Type=Inference-free Sparse Retriever2024.11 | 71.5 | — | — | — | — | |
| SPLADE-v3-DocFLOPS=1.42024.03 | 71.5 | — | — | — | — | |
| SPLADE-v3-LexicalFLOPS=0.62024.03 | 71.2 | — | — | — | — | |
| SPQEretriever=Dense, backbone=Contriever-msmarco2025.10 | 70.1 | — | — | — | — | |
| SPLADE-doc-distillRetriever Type=Inference-free Sparse Retriever, Evaluated by Current Authors=true2024.11 | 69.8 | 74.2 | — | — | — | |
| DeepImpactRetrieval Type=Sparse retrieval2021.09 | 69.5 | — | — | — | — | |
| OPQEretriever=Dense, backbone=Contriever-msmarco, model_scale=7B2025.10 | 69.5 | — | — | — | — | |
| SPLADE-maxRetrieval Type=Our methods2021.09 | 68.4 | 85.1 | — | — | — | |
| DRretriever=Dense, backbone=Contriever-msmarco, model_scale=7B2025.10 | 67.8 | — | — | — | — | |
| OPQEretriever=Dense, backbone=Contriever-msmarco, model_scale=3B2025.10 | 67.8 | — | — | — | — | |
| ANCEEvaluation Protocol=Rerank, Input Representation=FirstP2020.07 | 67.7 | — | — | — | — | |
| DRretriever=Dense, backbone=Contriever-msmarco, model_scale=3B2025.10 | 67.7 | — | — | — | — | |
| Baseretriever=Dense, backbone=Contriever-msmarco2025.10 | 67.5 | — | — | — | — | |
| ANCEEvaluation Protocol=Rerank, Input Representation=MaxP2020.07 | 67.1 | — | — | — | — | |
| SPLADE-docRetriever Type=Inference-free Sparse Retriever2024.11 | 66.7 | 94.7 | — | — | — | |
| SPLADE-docRetrieval Type=Our methods2021.09 | 66.7 | 74.7 | — | — | — | |
| SPLADERetrieval Type=Sparse retrieval2021.09 | 66.5 | 81.3 | — | — | — | |
| BM25 NegEvaluation Protocol=Rerank2020.07 | 66.4 | — | — | — | — | |
| COIL-tokRetrieval Type=Sparse retrieval2021.09 | 66 | — | — | — | — | |
| DPR (BM25 + Rand Neg)Evaluation Protocol=Rerank2020.07 | 65.3 | — | — | — | — | |
| ANCERetriever Type=Dense Retriever2024.11 | 64.8 | — | — | — | — | |
| BM25 -> BM25 + RandEvaluation Protocol=Rerank2020.07 | 64.8 | — | — | — | — | |
| ANCEEvaluation Protocol=Retrieval, Input Representation=FirstP2020.07 | 64.8 | — | — | — | — | |
| ANCERetrieval Type=Dense retrieval2021.09 | 64.8 | — | — | — | — | |
| BERT RerankerEvaluation Protocol=Rerank2020.07 | 64.6 | — | — | — | — | |
| doc2query-T5Retriever Type=Inference-free Sparse Retriever2024.11 | 64.2 | 82.7 | — | — | — | |
| doc2query-T5Retrieval Type=Sparse retrieval2021.09 | 64.2 | 82.7 | — | — | — | |
| ANCEEvaluation Protocol=Rerank, Input Representation=FirstP2020.07 | 64.1 | — | — | — | — | |
| BM25 -> NCE NegEvaluation Protocol=Rerank2020.07 | 63.8 | — | — | — | — | |
| BM25 -> RandEvaluation Protocol=Rerank2020.07 | 63.7 | — | — | — | — | |
| Siamese (ours)Retrieval Type=Dense retrieval2021.09 | 63.7 | 71.1 | — | — | — | |
| DPR (BM25 + Rand Neg)Evaluation Protocol=Rerank2020.07 | 62.9 | — | — | — | — | |
| ANCEEvaluation Protocol=Retrieval, Input Representation=MaxP2020.07 | 62.8 | — | — | — | — | |
| BM25 NegEvaluation Protocol=Rerank2020.07 | 62.6 | — | — | — | — | |
| BM25 -> BM25 + RandEvaluation Protocol=Rerank2020.07 | 62.6 | — | — | — | — | |
| NCE NegEvaluation Protocol=Rerank2020.07 | 61.8 | — | — | — | — | |
| Rand NegEvaluation Protocol=Rerank2020.07 | 61.5 | — | — | — | — | |
| ANCEEvaluation Protocol=Retrieval, Input Representation=FirstP2020.07 | 61.5 | — | — | — | — | |
| BM25 -> RandEvaluation Protocol=Rerank2020.07 | 60.9 | — | — | — | — | |
| BM25 -> NCE NegEvaluation Protocol=Rerank2020.07 | 60.8 | — | — | — | — | |
| Rand NegEvaluation Protocol=Rerank2020.07 | 60.5 | — | — | — | — | |
| NCE NegEvaluation Protocol=Rerank2020.07 | 60.2 | — | — | — | — | |
| DPR (BM25 + Rand Neg)Evaluation Protocol=Retrieval2020.07 | 60 | — | — | — | — | |
| SPQEretriever=Sparse BM252025.10 | 60 | — | — | — | — | |
| BM25 NegEvaluation Protocol=Retrieval2020.07 | 59.1 | — | — | — | — | |
| BM25 -> BM25 + RandEvaluation Protocol=Retrieval2020.07 | 59.1 | — | — | — | — | |
| BM25 -> RandEvaluation Protocol=Retrieval2020.07 | 57.6 | — | — | — | — | |
| BM25 -> NCE NegEvaluation Protocol=Retrieval2020.07 | 57.1 | — | — | — | — | |
| BM25 -> RandEvaluation Protocol=Retrieval2020.07 | 56.6 | — | — | — | — | |
| OPQEretriever=Sparse BM25, model_scale=7B2025.10 | 56.6 | — | — | — | — | |
| BM25 -> NCE NegEvaluation Protocol=Retrieval2020.07 | 56.4 | — | — | — | — | |
| DPR (BM25 + Rand Neg)Evaluation Protocol=Retrieval2020.07 | 55.7 | — | — | — | — | |
| Best TREC Trad RetrievalEvaluation Protocol=Retrieval2020.07 | 55.4 | — | — | — | — | |
| Best DeepCTEvaluation Protocol=Retrieval2020.07 | 55.4 | — | — | — | — | |
| Rand NegEvaluation Protocol=Retrieval2020.07 | 55.2 | — | — | — | — | |
| DeepCTRetriever Type=Inference-free Sparse Retriever2024.11 | 55.1 | 75.6 | — | — | — | |
| DeepCTRetrieval Type=Sparse retrieval2021.09 | 55.1 | 75.6 | — | — | — | |
| Best TREC Trad RetrievalEvaluation Protocol=Retrieval2020.07 | 54.9 | — | — | — | — | |
| Rand NegEvaluation Protocol=Retrieval2020.07 | 54.3 | — | — | — | — | |
| NCE NegEvaluation Protocol=Retrieval2020.07 | 54.2 | — | — | — | — | |
| BM25 -> BM25 + RandEvaluation Protocol=Retrieval2020.07 | 54 | — | — | — | — | |
| NCE NegEvaluation Protocol=Retrieval2020.07 | 53.9 | — | — | — | — | |
| OPQEretriever=Sparse BM25, model_scale=3B2025.10 | 53.9 | — | — | — | — | |
| BM25 NegEvaluation Protocol=Retrieval2020.07 | 52.9 | — | — | — | — | |
| BM25Evaluation Protocol=Retrieval2020.07 | 51.9 | — | — | — | — | |
| DRretriever=Sparse BM25, model_scale=7B2025.10 | 51.3 | — | — | — | — | |
| DRretriever=Sparse BM25, model_scale=3B2025.10 | 50.8 | — | — | — | — | |
| BM25Retriever Type=Inference-free Sparse Retriever2024.11 | 50.6 | 74.5 | — | — | — | |
| BM25Evaluation Protocol=Retrieval2020.07 | 50.6 | — | — | — | — | |
| BM25Retrieval Type=Sparse retrieval2021.09 | 50.6 | 74.5 | — | — | — | |
| Baseretriever=Sparse BM252025.10 | 50.6 | — | — | — | — | |
| BM25Evaluation Protocol=Zero-shot, Backbone=—, Variant=sparse, Fwd=—2026.05 | — | — | — | 50.6 | — | |
| DiffEmbedEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=single-vector, Fwd=12026.05 | — | — | 18.4 | — | — | |
| DiffEmbedEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=single-vector, Fwd=12026.05 | — | — | 11.1 | — | — | |
| DiffEmbedEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=single-vector, Fwd=12026.05 | — | — | 72 | — | — | |
| DiffEmbedEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=single-vector, Fwd=12026.05 | — | — | 69.5 | — | — | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=single-token, Fwd=12026.05 | — | — | 32.5 | 31.3 | 36.9 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=multi-token, Fwd=12026.05 | — | — | 47.9 | 42.7 | 49.6 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=single-token, Fwd=12026.05 | — | — | 39.1 | 45.1 | 48.1 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=multi-token, Fwd=12026.05 | — | — | 49 | 47.9 | 54.9 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=single-token, Fwd=12026.05 | — | — | 74.1 | 61.4 | 72.4 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=multi-token, Fwd=12026.05 | — | — | 75.1 | 62 | 73.9 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=single-token, Fwd=12026.05 | — | — | 65.6 | 56.2 | 63.7 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=multi-token, Fwd=12026.05 | — | — | 65.7 | 57.9 | 65.5 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=Qwen2.5, Variant=single-token, Fwd=12026.05 | — | — | 32 | 40.7 | 43.9 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=Qwen2.5, Variant=multi-token, Fwd=≤202026.05 | — | — | 40.4 | 40.5 | 47.7 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=LLaMA-3, Variant=single-token, Fwd=12026.05 | — | — | 51.1 | 43.1 | 51.3 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=LLaMA-3, Variant=multi-token, Fwd=≤202026.05 | — | — | 31.9 | 45.4 | 47.3 |