Document Reranking on BEIR (NDCG@10)
55.4Average NDCG@10DeBERTa-v3
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| DeBERTa-v3role=teacher2025.03 | 55.4 | — | — | — | — | — | — | — | — | |
| DiffuRank Perm-AssignBackbone=LLaDA-1.5-8B, Strategy=Perm-Assign, Fine-tuning dataset=MS MARCO2026.02 | 55.21 | 86.18 | 39.64 | 40.54 | 46.95 | 78.77 | 33 | 55.4 | 61.2 | |
| DiffuRank Perm-SampBackbone=LLaDA-1.5-8B, Strategy=Perm-Samp, Sampling Steps (K)=2, Fine-tuning dataset=MS MARCO2026.02 | 55.1 | 86.36 | 39.56 | 40.74 | 46.64 | 78.27 | 33.31 | 55.2 | 60.75 | |
| OSCAR-10-LayersDecoder=Mistral-7B2025.03 | 54.3 | — | — | — | — | — | — | — | — | |
| Llama-3.1-8B ListwiseBackbone=Llama-3.1-8B, Strategy=Listwise, Fine-tuning dataset=MS MARCO2026.02 | 54.28 | 87.6 | 39.84 | 34.64 | 46.21 | 78.59 | 34.16 | 52.43 | 60.74 | |
| DiffuRank Logits-ListBackbone=LLaDA-1.5-8B, Strategy=Logits-List, Fine-tuning dataset=MS MARCO2026.02 | 54.09 | 86.91 | 39.22 | 37.78 | 45.9 | 77.42 | 32.62 | 52.29 | 60.59 | |
| Qwen3-8B PointwiseBackbone=Qwen3-8B, Strategy=Pointwise, Fine-tuning dataset=MS MARCO2026.02 | 53.92 | 85.64 | 40.41 | 35.8 | 45.27 | 79.33 | 29.31 | 49.17 | 66.41 | |
| Qwen3-8B ListwiseBackbone=Qwen3-8B, Strategy=Listwise, Fine-tuning dataset=MS MARCO2026.02 | 53.39 | 85.37 | 40.05 | 31.73 | 45.44 | 78.96 | 32.48 | 52.36 | 60.72 | |
| OSCAR-llamaDecoder=Mistral-7B2025.03 | 52.8 | — | — | — | — | — | — | — | — | |
| DiffuRank PointwiseBackbone=LLaDA-1.5-8B, Strategy=Pointwise, Fine-tuning dataset=MS MARCO2026.02 | 52.68 | 86.72 | 39.8 | 31.48 | 45.45 | 75.37 | 30.24 | 52.17 | 60.23 | |
| OSCAR-8-LayersDecoder=Mistral-7B2025.03 | 52.5 | — | — | — | — | — | — | — | — | |
| LLaDA-1.5-8B ListwiseBackbone=LLaDA-1.5-8B, Strategy=Listwise, Fine-tuning dataset=MS MARCO2026.02 | 52.48 | 80.31 | 38.67 | 40.2 | 43.44 | 77.28 | 31.93 | 51.53 | 56.47 | |
| Llama-3.1-8B PointwiseBackbone=Llama-3.1-8B, Strategy=Pointwise, Fine-tuning dataset=MS MARCO2026.02 | 51.68 | 83.49 | 35.47 | 33.52 | 41.13 | 79.59 | 29.63 | 47.85 | 62.73 | |
| RankZephyr-7B ListwiseBackbone=Zephyr-7B, Strategy=Listwise, Fine-tuning dataset=MS MARCO2026.02 | 51.15 | 82.92 | 38.26 | 30.22 | 44.42 | 75.42 | 31.41 | 52.8 | 53.73 | |
| monoT5 (220M)Backbone=T5, Fine-tuning dataset=MS MARCO2026.02 | 46.44 | 75.94 | 35.42 | 30.35 | 42.43 | 65.07 | 31.2 | 46.98 | 44.15 | |
| monoBERT (340M)Backbone=BERT, Fine-tuning dataset=MS MARCO2026.02 | 45.55 | 73.45 | 34.92 | 30.26 | 41.69 | 62.22 | 30.63 | 47.03 | 44.21 | |
| BM25Type=Initial Retrieval2026.02 | 43.43 | 59.47 | 33.75 | 44.22 | 31.8 | 67.89 | 33.05 | 39.52 | 40.7 |