Document Reranking on BEIR
85.51NDCG@10 (Covid)RankGPT-4^
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| RankGPT-4^Model=GPT-4, Evaluation Protocol=Zero-Shot, Reranking Depth=top-30, Note=GPT-3.5 pre-rerank -> GPT-42026.04 | 85.51 | 38.47 | 38.57 | 47.12 | 74.95 | — | — | 57.55 | 53.68 | 34.4 | |
| RankGPT (GPT-4)Model=GPT-4, Evaluation Protocol=Zero-Shot, Reranking Depth=top-100, Note=direct rerank on top-100 BM252026.04 | 84.92 | 39.05 | 36.47 | 45.79 | 77.61 | — | — | 60.98 | 53.78 | 34.2 | |
| BracketRank (GPT-4) -20Model=GPT-4, Evaluation Protocol=Zero-Shot, Reranking Depth=202026.04 | 84.82 | 40.86 | 39.67 | 45.58 | 78.03 | — | — | 61.94 | 54.66 | 33.49 | |
| ListT5 (3B)Model=3B, Evaluation Protocol=Supervised2026.04 | 84.7 | 37.7 | 33.6 | 46.2 | 77 | — | — | 57.8 | 53 | 33.8 | |
| TourRank (GPT-4)Model=GPT-4, Evaluation Protocol=Zero-Shot2026.04 | 82.59 | 37.99 | 29.98 | 44.64 | 72.17 | — | — | 57.87 | 50.94 | 30.83 | |
| Cohere Rerank-v2Evaluation Protocol=Supervised2026.04 | 81.81 | 36.36 | 32.51 | 42.51 | 74.44 | — | — | 50.78 | 49.45 | 29.6 | |
| DiffuRank Logits-ListBackbone=LLaDA-1.5-8B, Strategy=Logits-List, Protocol=Zero-shot2026.02 | 81.08 | 34.08 | 31.4 | 39.79 | 70.52 | 29.81 | 48.74 | 50.75 | 48.27 | — | |
| monoT5 (3B)Model=T5 (3B), Evaluation Protocol=Supervised2026.04 | 80.71 | 38.97 | 32.41 | 44.45 | 76.57 | — | — | 56.71 | 51.36 | 32.55 | |
| Qwen2-8B ListwiseBackbone=Qwen2-8B, Strategy=Listwise, Protocol=Zero-shot2026.02 | 80.66 | 38.28 | 35.94 | 42.18 | 75.91 | 30.56 | 48.86 | 54.87 | 50.91 | — | |
| BracketRank (GPT-4) -15Model=GPT-4, Evaluation Protocol=Zero-Shot, Reranking Depth=152026.04 | 80.46 | 40.04 | 35.94 | 43.95 | 77.91 | — | — | 59.57 | 52.58 | 32.91 | |
| DiffuRank PointwiseBackbone=LLaDA-1.5-8B, Strategy=Pointwise, Protocol=Zero-shot2026.02 | 79.27 | 35.63 | 23.02 | 31.6 | 61.95 | 28.04 | 39.32 | 53.31 | 44.02 | — | |
| Llama-3.1-8B PointwiseBackbone=Llama-3.1-8B, Strategy=Pointwise, Protocol=Zero-shot2026.02 | 78.59 | 34.93 | 29.08 | 32.43 | 61.31 | 23.84 | 33.99 | 62.16 | 44.54 | — | |
| monoT5 (220M)Model=T5 (220M), Evaluation Protocol=Supervised2026.04 | 78.34 | 37.38 | 30.82 | 42.42 | 73.4 | — | — | 51.72 | 49.07 | 31.67 | |
| RankGPT-3.5Model=GPT-3.5, Evaluation Protocol=Zero-Shot2026.04 | 76.67 | 35.62 | 36.18 | 44.47 | 70.43 | — | — | 50.62 | 49.37 | 32.12 | |
| Llama-3.1-8B ListwiseBackbone=Llama-3.1-8B, Strategy=Listwise, Protocol=Zero-shot2026.02 | 76.49 | 36.53 | 35.13 | 41.36 | 69.48 | 31.7 | 46.19 | 51.22 | 48.51 | — | |
| Promptagator++Evaluation Protocol=Zero-Shot2026.04 | 76.2 | 37 | 38.1 | 43.4 | 73.1 | — | — | — | — | — | |
| Qwen2-8B PointwiseBackbone=Qwen2-8B, Strategy=Pointwise, Protocol=Zero-shot2026.02 | 74.82 | 35.05 | 25.47 | 32.86 | 68.28 | 25.98 | 36.85 | 61.57 | 45.11 | — | |
| DiffuRank Perm-SampBackbone=LLaDA-1.5-8B, Strategy=Permutation-based Sampling, Sampling step (K)=2, Protocol=Zero-shot2026.02 | 73.08 | 35.48 | 36.17 | 43.34 | 72.9 | 34.17 | 46.3 | 49.16 | 48.83 | — | |
| DiffuRank Perm-AssignBackbone=LLaDA-1.5-8B, Strategy=Permutation-based Assignment, Protocol=Zero-shot2026.02 | 72.99 | 36.08 | 37.18 | 43.34 | 72 | 34.61 | 46.79 | 51.38 | 49.3 | — | |
| monoBERT (340M)Model=340M, Evaluation Protocol=Supervised2026.04 | 70.01 | 36.88 | 31.75 | 41.87 | 71.36 | — | — | 49.35 | 47.16 | 31.44 | |
| UPR (3B)Model=3B, Evaluation Protocol=Zero-Shot2026.04 | 68.11 | 35.04 | 19.69 | 30.91 | 72.69 | — | — | 42.43 | 42.99 | 31.91 | |
| LLaDA-1.5-8B ListwiseBackbone=LLaDA-1.5-8B, Strategy=Listwise, Protocol=Zero-shot2026.02 | 64.49 | 35.09 | 37.09 | 38.94 | 71.26 | 32.4 | 43.33 | 45.54 | 46.02 | — | |
| BM25Protocol=Zero-shot, Top Documents=1002026.02 | 59.47 | 33.75 | 44.22 | 31.8 | 67.89 | 33.05 | 39.52 | 40.7 | 43.43 | — | |
| BM252026.04 | 59.47 | 30.75 | 44.22 | 31.8 | 67.89 | — | — | 40.7 | 43.42 | 33.05 |