Reranking on TREC DL 2019 (test)
71.52NDCG@10HeapSort
Evaluation results
| Method | Links | |
|---|---|---|
| HeapSortNumber of LLM calls (budget)=300, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 71.52 | |
| HeapSortNumber of LLM calls (budget)=450, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 71.14 | |
| MohajerNumber of LLM calls (budget)=250, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.98 | |
| MohajerNumber of LLM calls (budget)=300, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.98 | |
| MohajerNumber of LLM calls (budget)=350, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.98 | |
| MohajerNumber of LLM calls (budget)=400, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.98 | |
| MohajerNumber of LLM calls (budget)=450, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.98 | |
| MohajerNumber of LLM calls (budget)=500, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.98 | |
| HeapSortNumber of LLM calls (budget)=400, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.73 | |
| HeapSortNumber of LLM calls (budget)=500, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.7 | |
| Mohajer + BubbleNumber of LLM calls (budget)=450, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 70.49 | |
| Mohajer + BubbleNumber of LLM calls (budget)=500, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 70.49 | |
| Mohajer + BubbleNumber of LLM calls (budget)=350, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.44 | |
| MohajerNumber of LLM calls (budget)=450, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 70.43 | |
| MohajerNumber of LLM calls (budget)=500, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 70.43 | |
| Mohajer + BubbleNumber of LLM calls (budget)=300, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.33 | |
| Mohajer + BubbleNumber of LLM calls (budget)=400, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.26 | |
| Mohajer + BubbleNumber of LLM calls (budget)=450, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.26 | |
| Mohajer + BubbleNumber of LLM calls (budget)=500, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 70.26 | |
| Mohajer + BubbleNumber of LLM calls (budget)=400, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 70.08 | |
| MohajerNumber of LLM calls (budget)=400, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 70.08 | |
| HeapSortNumber of LLM calls (budget)=350, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 69.61 | |
| Mohajer + BubbleNumber of LLM calls (budget)=350, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 69.39 | |
| MohajerNumber of LLM calls (budget)=350, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 69.39 | |
| Mohajer + BubbleNumber of LLM calls (budget)=200, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 69.37 | |
| MohajerNumber of LLM calls (budget)=200, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 69.37 | |
| HeapSortNumber of LLM calls (budget)=500, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 69.14 | |
| Mohajer + BubbleNumber of LLM calls (budget)=250, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 68.82 | |
| Mohajer + BubbleNumber of LLM calls (budget)=300, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 68.6 | |
| MohajerNumber of LLM calls (budget)=300, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 68.6 | |
| HeapSortNumber of LLM calls (budget)=250, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 68.15 | |
| BubbleSortNumber of LLM calls (budget)=500, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 67.12 | |
| Mohajer + BubbleNumber of LLM calls (budget)=250, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 66.69 | |
| MohajerNumber of LLM calls (budget)=250, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 66.69 | |
| Mohajer + BubbleNumber of LLM calls (budget)=150, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 66.49 | |
| MohajerNumber of LLM calls (budget)=150, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 66.49 | |
| BubbleSortNumber of LLM calls (budget)=400, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 66.44 | |
| BubbleSortNumber of LLM calls (budget)=450, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 66.27 | |
| QuickSortNumber of LLM calls (budget)=500, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 64.81 | |
| BubbleSortNumber of LLM calls (budget)=300, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 64.32 | |
| PAC + BubbleNumber of LLM calls (budget)=350, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 64.25 | |
| PAC + BubbleNumber of LLM calls (budget)=400, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 64.25 | |
| PAC + BubbleNumber of LLM calls (budget)=450, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 64.25 | |
| PAC + BubbleNumber of LLM calls (budget)=500, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 64.25 | |
| HeapSortNumber of LLM calls (budget)=450, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 64.24 | |
| BubbleSortNumber of LLM calls (budget)=350, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 64.24 | |
| Mohajer + BubbleNumber of LLM calls (budget)=100, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.37 | |
| MohajerNumber of LLM calls (budget)=100, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.37 | |
| PAC + BubbleNumber of LLM calls (budget)=200, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.02 | |
| PAC + BubbleNumber of LLM calls (budget)=250, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.02 | |
| PAC + BubbleNumber of LLM calls (budget)=300, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.02 | |
| PAC + BubbleNumber of LLM calls (budget)=350, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.02 | |
| PAC + BubbleNumber of LLM calls (budget)=400, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.02 | |
| PAC + BubbleNumber of LLM calls (budget)=450, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.02 | |
| PAC + BubbleNumber of LLM calls (budget)=500, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 63.02 | |
| Mohajer + BubbleNumber of LLM calls (budget)=200, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 62.97 | |
| MohajerNumber of LLM calls (budget)=200, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 62.97 | |
| QuickSortNumber of LLM calls (budget)=400, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 62.13 | |
| QuickSortNumber of LLM calls (budget)=450, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 61.92 | |
| BubbleSortNumber of LLM calls (budget)=500, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 61.9 | |
| BubbleSortNumber of LLM calls (budget)=250, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 61.45 | |
| BubbleSortNumber of LLM calls (budget)=450, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 61.3 | |
| BubbleSortNumber of LLM calls (budget)=200, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 61.11 | |
| BubbleSortNumber of LLM calls (budget)=400, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 60.98 | |
| QuickSortNumber of LLM calls (budget)=350, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 60.92 | |
| PAC + BubbleNumber of LLM calls (budget)=300, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 60.74 | |
| QuickSortNumber of LLM calls (budget)=300, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 59.96 | |
| QuickSortNumber of LLM calls (budget)=500, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 59.67 | |
| QuickSortNumber of LLM calls (budget)=450, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 59.09 | |
| BubbleSortNumber of LLM calls (budget)=350, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.73 | |
| QuickSortNumber of LLM calls (budget)=300, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.71 | |
| QuickSortNumber of LLM calls (budget)=350, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.71 | |
| QuickSortNumber of LLM calls (budget)=400, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.71 | |
| QuickSortNumber of LLM calls (budget)=250, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.63 | |
| QuickSortNumber of LLM calls (budget)=100, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.28 | |
| QuickSortNumber of LLM calls (budget)=200, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 58.25 | |
| QuickSortNumber of LLM calls (budget)=150, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.11 | |
| QuickSortNumber of LLM calls (budget)=200, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 58.04 | |
| BubbleSortNumber of LLM calls (budget)=150, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 57.79 | |
| BubbleSortNumber of LLM calls (budget)=100, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 57.49 | |
| BubbleSortNumber of LLM calls (budget)=300, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 57.18 | |
| BubbleSortNumber of LLM calls (budget)=200, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 57.04 | |
| BubbleSortNumber of LLM calls (budget)=250, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 57.04 | |
| QuickSortNumber of LLM calls (budget)=250, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 57.03 | |
| PAC + BubbleNumber of LLM calls (budget)=150, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 57 | |
| QuickSortNumber of LLM calls (budget)=150, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 56.56 | |
| HeapSortNumber of LLM calls (budget)=400, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 55.85 | |
| QuickSortNumber of LLM calls (budget)=100, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 54.94 | |
| HeapSortNumber of LLM calls (budget)=200, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 53.18 | |
| BubbleSortNumber of LLM calls (budget)=100, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 50.58 | |
| PAC + BubbleNumber of LLM calls (budget)=100, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 50.58 | |
| BubbleSortNumber of LLM calls (budget)=150, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 50.58 | |
| PAC + BubbleNumber of LLM calls (budget)=150, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 50.58 | |
| PAC + BubbleNumber of LLM calls (budget)=200, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 50.58 | |
| PAC + BubbleNumber of LLM calls (budget)=250, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 50.58 | |
| PAC + BubbleNumber of LLM calls (budget)=100, Comparison strategy=Randomized, LLM Model=Qwen3-4B-Instruct2026.05 | 50.58 | |
| HeapSortNumber of LLM calls (budget)=350, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 42.42 | |
| Mohajer + BubbleNumber of LLM calls (budget)=100, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 32.76 | |
| MohajerNumber of LLM calls (budget)=100, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 32.76 | |
| Mohajer + BubbleNumber of LLM calls (budget)=150, Comparison strategy=Bidirectional, LLM Model=Qwen3-4B-Instruct2026.05 | 32.76 |