Retrieval on MS MARCO (MRR@10 components)
0.433D (MRR@10)DiffRetriever
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=multi-token, Fwd=12026.05 | 0.433 | 0.349 | 0.411 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=LLaMA-3, Variant=multi-token, Fwd=42026.05 | 0.43 | 0.348 | 0.414 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=multi-token, Fwd=12026.05 | 0.427 | 0.348 | 0.408 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=LLaMA-3, Variant=single-token, Fwd=12026.05 | 0.425 | 0.347 | 0.41 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=Qwen2.5, Variant=multi-token, Fwd=42026.05 | 0.424 | 0.347 | 0.406 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=single-token, Fwd=12026.05 | 0.424 | 0.341 | 0.405 | |
| DiffRetrieverEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=single-token, Fwd=12026.05 | 0.424 | 0.347 | 0.405 | |
| PromptRepsEvaluation Protocol=Fine-tuned, Backbone=Qwen2.5, Variant=single-token, Fwd=12026.05 | 0.419 | 0.343 | 0.405 | |
| RepLLaMAEvaluation Protocol=Fine-tuned, Backbone=LLaMA, Variant=single-vector, Fwd=12026.05 | 0.412 | — | — | |
| DiffEmbedEvaluation Protocol=Fine-tuned, Backbone=Dream, Variant=single-vector, Fwd=12026.05 | 0.405 | — | — | |
| DiffEmbedEvaluation Protocol=Fine-tuned, Backbone=LLaDA, Variant=single-vector, Fwd=12026.05 | 0.398 | — | — | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=multi-token, Fwd=12026.05 | 0.192 | 0.165 | 0.218 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=multi-token, Fwd=12026.05 | 0.192 | 0.223 | 0.248 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=LLaMA-3, Variant=single-token, Fwd=12026.05 | 0.178 | 0.211 | 0.242 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=single-token, Fwd=12026.05 | 0.158 | 0.2 | 0.221 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=LLaMA-3, Variant=multi-token, Fwd=≤202026.05 | 0.151 | 0.196 | 0.22 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=Qwen2.5, Variant=multi-token, Fwd=≤202026.05 | 0.142 | 0.177 | 0.2 | |
| PromptRepsEvaluation Protocol=Zero-shot, Backbone=Qwen2.5, Variant=single-token, Fwd=12026.05 | 0.1 | 0.19 | 0.2 | |
| DiffRetrieverEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=single-token, Fwd=12026.05 | 0.086 | 0.073 | 0.112 | |
| DiffEmbedEvaluation Protocol=Zero-shot, Backbone=Dream, Variant=single-vector, Fwd=12026.05 | 0.034 | — | — | |
| DiffEmbedEvaluation Protocol=Zero-shot, Backbone=LLaDA, Variant=single-vector, Fwd=12026.05 | 0.014 | — | — | |
| BM25Evaluation Protocol=Zero-shot, Backbone=—, Variant=sparse, Fwd=—2026.05 | — | 0.184 | — |