Document Reranking on TREC DL 2020
70.86NDCG@10RankZephyr
Evaluation results
| Method | Links | |
|---|---|---|
| RankZephyrRet.=8×BM252026.04 | 70.86 | |
| RankGPT-4Ret.=3×BM252026.04 | 70.56 | |
| TourRankRet.=3×BM252026.04 | 69.56 | |
| ListT5-3BRet.=8×BM252026.04 | 69.1 | |
| ResRankswRet.=8×BM252026.04 | 69.06 | |
| monoT5Ret.=2×BM252026.04 | 68.89 | |
| ResRankRet.=8×BM252026.04 | 68.65 | |
| ListT5-baseRet.=8×BM252026.04 | 68.1 | |
| RankMistralRet.=8×BM252026.04 | 68.07 | |
| monoBERTRet.=2×BM252026.04 | 67.28 | |
| RankVicunaRet.=8×BM252026.04 | 65.49 | |
| PE-RankRet.=8×BM252026.04 | 63.54 | |
| RankGPT-3.5Ret.=3×BM252026.04 | 62.91 | |
| BM25Ret.=–2026.04 | 47.96 | |
| BubbleSortBase Model=Qwen3-4B-Instruct, Comparison Mode=Bidirectional2026.05 | 0.7045 | |
| HeapSortBase Model=Qwen3-4B-Instruct, Comparison Mode=Bidirectional2026.05 | 0.6946 | |
| QuickSortBase Model=Qwen3-4B-Instruct, Comparison Mode=Bidirectional2026.05 | 0.6883 | |
| HeapSortNumber of LLM calls (budget)=500, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6858 | |
| HeapSortNumber of LLM calls (budget)=450, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6778 | |
| Mohajer + BubbleNumber of LLM calls (budget)=200, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6766 | |
| MohajerNumber of LLM calls (budget)=200, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6766 | |
| HeapSortNumber of LLM calls (budget)=350, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6764 | |
| MohajerNumber of LLM calls (budget)=250, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6762 | |
| MohajerNumber of LLM calls (budget)=300, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6762 | |
| MohajerNumber of LLM calls (budget)=350, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6762 | |
| MohajerNumber of LLM calls (budget)=400, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6762 | |
| MohajerNumber of LLM calls (budget)=450, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6762 | |
| MohajerNumber of LLM calls (budget)=500, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6762 | |
| Mohajer + BubbleNumber of LLM calls (budget)=250, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6761 | |
| Mohajer + BubbleNumber of LLM calls (budget)=350, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.676 | |
| Mohajer + BubbleBase Model=Qwen3-4B-Instruct, Comparison Mode=Randomized2026.05 | 0.6758 | |
| HeapSortNumber of LLM calls (budget)=500, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.675 | |
| Mohajer + BubbleNumber of LLM calls (budget)=450, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6745 | |
| Mohajer + BubbleNumber of LLM calls (budget)=500, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6745 | |
| MohajerNumber of LLM calls (budget)=450, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6735 | |
| MohajerNumber of LLM calls (budget)=500, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6735 | |
| HeapSortNumber of LLM calls (budget)=300, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6731 | |
| HeapSortNumber of LLM calls (budget)=400, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6727 | |
| PAC + BubbleBase Model=Qwen3-4B-Instruct, Comparison Mode=Bidirectional2026.05 | 0.6709 | |
| Mohajer + BubbleNumber of LLM calls (budget)=300, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6705 | |
| Mohajer + BubbleNumber of LLM calls (budget)=400, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6702 | |
| MohajerNumber of LLM calls (budget)=400, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6698 | |
| Mohajer + BubbleBase Model=Qwen3-4B-Instruct, Comparison Mode=Bidirectional2026.05 | 0.6692 | |
| MohajerBase Model=Qwen3-4B-Instruct, Comparison Mode=Bidirectional2026.05 | 0.6692 | |
| Mohajer + BubbleNumber of LLM calls (budget)=350, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6632 | |
| MohajerNumber of LLM calls (budget)=350, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6632 | |
| Mohajer + BubbleNumber of LLM calls (budget)=400, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.663 | |
| Mohajer + BubbleNumber of LLM calls (budget)=450, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.663 | |
| Mohajer + BubbleNumber of LLM calls (budget)=500, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.663 | |
| PAC + BubbleBase Model=Qwen3-4B-Instruct, Comparison Mode=Randomized2026.05 | 0.6605 | |
| BubbleSortNumber of LLM calls (budget)=500, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6588 | |
| Mohajer + BubbleNumber of LLM calls (budget)=300, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6583 | |
| MohajerNumber of LLM calls (budget)=300, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6583 | |
| MohajerBase Model=Qwen3-4B-Instruct, Comparison Mode=Randomized2026.05 | 0.6543 | |
| Mohajer + BubbleNumber of LLM calls (budget)=150, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6506 | |
| MohajerNumber of LLM calls (budget)=150, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6506 | |
| HeapSortNumber of LLM calls (budget)=250, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6488 | |
| BubbleSortNumber of LLM calls (budget)=450, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6437 | |
| Mohajer + BubbleNumber of LLM calls (budget)=250, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6425 | |
| MohajerNumber of LLM calls (budget)=250, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6425 | |
| BubbleSortNumber of LLM calls (budget)=400, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6335 | |
| Mohajer + BubbleNumber of LLM calls (budget)=200, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6224 | |
| MohajerNumber of LLM calls (budget)=200, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6224 | |
| HeapSortNumber of LLM calls (budget)=450, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6205 | |
| QuickSortNumber of LLM calls (budget)=500, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6191 | |
| BubbleSortNumber of LLM calls (budget)=350, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.6178 | |
| BubbleSortNumber of LLM calls (budget)=300, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.613 | |
| BubbleSortNumber of LLM calls (budget)=500, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.6007 | |
| Mohajer + BubbleNumber of LLM calls (budget)=100, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5995 | |
| MohajerNumber of LLM calls (budget)=100, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5995 | |
| BubbleSortNumber of LLM calls (budget)=450, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.599 | |
| BubbleSortNumber of LLM calls (budget)=400, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5986 | |
| QuickSortNumber of LLM calls (budget)=450, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5976 | |
| BubbleSortNumber of LLM calls (budget)=200, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5903 | |
| PAC + BubbleNumber of LLM calls (budget)=400, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5862 | |
| PAC + BubbleNumber of LLM calls (budget)=450, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5862 | |
| PAC + BubbleNumber of LLM calls (budget)=500, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5862 | |
| PAC + BubbleNumber of LLM calls (budget)=350, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5859 | |
| PAC + BubbleNumber of LLM calls (budget)=200, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5851 | |
| PAC + BubbleNumber of LLM calls (budget)=250, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5851 | |
| PAC + BubbleNumber of LLM calls (budget)=300, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5851 | |
| PAC + BubbleNumber of LLM calls (budget)=350, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5851 | |
| PAC + BubbleNumber of LLM calls (budget)=400, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5851 | |
| PAC + BubbleNumber of LLM calls (budget)=450, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5851 | |
| PAC + BubbleNumber of LLM calls (budget)=500, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5851 | |
| BubbleSortNumber of LLM calls (budget)=250, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5845 | |
| QuickSortNumber of LLM calls (budget)=400, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5725 | |
| BubbleSortNumber of LLM calls (budget)=350, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5618 | |
| QuickSortNumber of LLM calls (budget)=300, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5615 | |
| QuickSortNumber of LLM calls (budget)=350, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5605 | |
| BubbleSortNumber of LLM calls (budget)=200, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5572 | |
| BubbleSortNumber of LLM calls (budget)=250, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5572 | |
| BubbleSortNumber of LLM calls (budget)=300, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5572 | |
| BubbleSortNumber of LLM calls (budget)=150, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5531 | |
| PAC + BubbleNumber of LLM calls (budget)=150, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5527 | |
| BubbleSortNumber of LLM calls (budget)=100, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5523 | |
| PAC + BubbleNumber of LLM calls (budget)=300, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5504 | |
| QuickSortNumber of LLM calls (budget)=250, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5467 | |
| HeapSortNumber of LLM calls (budget)=400, Prompting strategy=Bidirectional, Backbone=Flan-T5-XL2026.05 | 0.5444 | |
| QuickSortNumber of LLM calls (budget)=200, Prompting strategy=Randomized, Backbone=Flan-T5-XL2026.05 | 0.5434 |