Information Retrieval on TREC DL 2020
75.4nDCG@10SPLADE-v3
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| SPLADE-v3FLOPS=1.22024.03 | 75.4 | — | — | — | — | — | |
| SPLADE-v3-DistilBERTFLOPS=1.42024.03 | 74.4 | — | — | — | — | — | |
| SPLADE-v3-LexicalFLOPS=0.62024.03 | 73.6 | — | — | — | — | — | |
| SPLADE++SDFLOPS=1.42024.03 | 71.8 | — | — | — | — | — | |
| OPQEretriever=Dense, backbone=Contriever-msmarco, model_scale=7B2025.10 | 70.5 | — | — | — | — | — | |
| SPLADE-v3-DocFLOPS=1.42024.03 | 70.3 | — | — | — | — | — | |
| SPQEretriever=Dense, backbone=Contriever-msmarco2025.10 | 69.3 | — | — | — | — | — | |
| STORMBackbone=Qwen3-8B, Max tokens=642026.06 | 67.3 | — | — | — | — | — | |
| OPQEretriever=Dense, backbone=Contriever-msmarco, model_scale=3B2025.10 | 66.8 | — | — | — | — | — | |
| Baseretriever=Dense, backbone=Contriever-msmarco2025.10 | 66.6 | — | — | — | — | — | |
| DRretriever=Dense, backbone=Contriever-msmarco, model_scale=3B2025.10 | 66.6 | — | — | — | — | — | |
| DRretriever=Dense, backbone=Contriever-msmarco, model_scale=7B2025.10 | 66.5 | — | — | — | — | — | |
| Q2DBackbone=GPT-4.1, Evaluation Protocol=Zero-shot2026.06 | 66.2 | — | — | — | — | — | |
| MUGIBackbone=GPT-4.12026.06 | 65.8 | — | — | — | — | — | |
| CSQEBackbone=GPT-4.12026.06 | 65.5 | — | — | — | — | — | |
| MUGI + ReFormeREnhancement=ReFormeR2026.04 | 65.3 | 43.2 | 90.1 | — | — | — | |
| LameRBackbone=GPT-4.12026.06 | 65.3 | — | — | — | — | — | |
| QA-ExpaBackbone=GPT-4.12026.06 | 64.2 | — | — | — | — | — | |
| MUGI2026.04 | 63.9 | 41.7 | 88.7 | — | — | — | |
| Query2Doc (FS) + ReFormeRPrompting Strategy=Few-shot, Enhancement=ReFormeR2026.04 | 62.6 | 41.3 | 89 | — | — | — | |
| Query2Doc (ZS) + ReFormeRPrompting Strategy=Zero-shot, Enhancement=ReFormeR2026.04 | 61.8 | 41 | 88.3 | — | — | — | |
| Query2Doc (CoT) + ReFormeRPrompting Strategy=Chain-of-Thought, Enhancement=ReFormeR2026.04 | 61.1 | 41.3 | 88.5 | — | — | — | |
| Query2Doc (FS)Prompting Strategy=Few-shot2026.04 | 60.4 | 39.4 | 87.6 | — | — | — | |
| SPQEretriever=Sparse BM252025.10 | 60.1 | — | — | — | — | — | |
| Query2Doc (CoT)Prompting Strategy=Chain-of-Thought2026.04 | 59.5 | 37.2 | 87.5 | — | — | — | |
| QA-Expand + ReFormeREnhancement=ReFormeR2026.04 | 57.9 | 38.2 | 88 | — | — | — | |
| Query2Doc (ZS)Prompting Strategy=Zero-shot2026.04 | 57.9 | 37.9 | 88.1 | — | — | — | |
| Q2KBackbone=GPT-4.12026.06 | 57.6 | — | — | — | — | — | |
| ReFormeR2026.04 | 57.2 | 33.9 | 84.5 | — | — | — | |
| QA-Expand2026.04 | 56.7 | 37.6 | 84.7 | — | — | — | |
| OPQEretriever=Sparse BM25, model_scale=7B2025.10 | 55.4 | — | — | — | — | — | |
| GenQREBackbone=GPT-4.12026.06 | 55.3 | — | — | — | — | — | |
| GenQR2026.04 | 55.1 | 33.6 | 84 | — | — | — | |
| GenQRBackbone=GPT-4.12026.06 | 53.7 | — | — | — | — | — | |
| OPQEretriever=Sparse BM25, model_scale=3B2025.10 | 51.3 | — | — | — | — | — | |
| DRretriever=Sparse BM25, model_scale=7B2025.10 | 49.9 | — | — | — | — | — | |
| DRretriever=Sparse BM25, model_scale=3B2025.10 | 49.4 | — | — | — | — | — | |
| BM25 + RM3Enhancement=RM32026.04 | 49.2 | 30.2 | 82.9 | — | — | — | |
| BM25 + RocchioEnhancement=Rocchio2026.04 | 49.1 | 31.2 | 81.6 | — | — | — | |
| RM3Backbone=GPT-4.12026.06 | 49 | — | — | — | — | — | |
| BM252026.04 | 48.8 | 28.8 | 80.3 | — | — | — | |
| Baseretriever=Sparse BM252025.10 | 48 | — | — | — | — | — | |
| GenQREnsemble2026.04 | 45.3 | 28.7 | 79.5 | — | — | — | |
| BM25Evaluation Protocol=Zero-shot, Backbone=—, Variant=sparse, Fwd=—2026.05 | — | — | — | — | 48 | — | |
| DiffEmbedEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=single-vector, Fwd=12026.05 | — | — | — | 11 | — | — | |
| DiffEmbedEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=single-vector, Fwd=12026.05 | — | — | — | 6.3 | — | — | |
| DiffEmbedEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=single-vector, Fwd=12026.05 | — | — | — | 69.3 | — | — | |
| DiffEmbedEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=single-vector, Fwd=12026.05 | — | — | — | 67.6 | — | — | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=single-token, Fwd=12026.05 | — | — | — | 25.5 | 20.8 | 31.2 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=multi-token, Fwd=12026.05 | — | — | — | 46.6 | 42.2 | 51.2 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=single-token, Fwd=12026.05 | — | — | — | 36.7 | 46.3 | 48.3 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=multi-token, Fwd=12026.05 | — | — | — | 47.2 | 47.9 | 53.6 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=single-token, Fwd=12026.05 | — | — | — | 72.1 | 62.7 | 69 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=multi-token, Fwd=12026.05 | — | — | — | 72.9 | 61.7 | 69.7 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=single-token, Fwd=12026.05 | — | — | — | 71.5 | 62.4 | 70.1 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=multi-token, Fwd=12026.05 | — | — | — | 72.1 | 61.4 | 69.8 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=Qwen2.5, Variant=single-token, Fwd=12026.05 | — | — | — | 24.5 | 42.8 | 42.2 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=Qwen2.5, Variant=multi-token, Fwd=≤202026.05 | — | — | — | 38.6 | 41 | 47.8 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=LLaMA-3, Variant=single-token, Fwd=12026.05 | — | — | — | 44.5 | 45.2 | 53.6 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=LLaMA-3, Variant=multi-token, Fwd=≤202026.05 | — | — | — | 33.6 | 41.2 | 47.1 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=Qwen2.5, Variant=single-token, Fwd=12026.05 | — | — | — | 71.5 | 62 | 70.3 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=Qwen2.5, Variant=multi-token, Fwd=42026.05 | — | — | — | 73.1 | 63.2 | 70.5 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=LLaMA-3, Variant=single-token, Fwd=12026.05 | — | — | — | 75.1 | 63.1 | 70.7 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=LLaMA-3, Variant=multi-token, Fwd=42026.05 | — | — | — | 73.9 | 62.1 | 70.9 | |
| RepLLaMAEvaluation Protocol=Fine-tuned, Backbone=LLaMA, Variant=single-vector, Fwd=12026.05 | — | — | — | 69 | — | — |