Document Reranking on TREC DL 2019 and 2020 (test)
68.71NDCG@10HeapSort
Evaluation results
| Method | Links | |
|---|---|---|
| HeapSortOracle=Randomized, Budget=500, Model=Flan-T5-XL2026.05 | 68.71 | |
| HeapSortOracle=Randomized, Budget=450, Model=Flan-T5-XL2026.05 | 68.53 | |
| HeapSortOracle=Randomized, Budget=300, Model=Flan-T5-XL2026.05 | 68.5 | |
| HeapSortOracle=Randomized, Budget=350, Model=Flan-T5-XL2026.05 | 68.41 | |
| HeapSortOracle=Randomized, Budget=400, Model=Flan-T5-XL2026.05 | 68.34 | |
| Mohajer + BubbleOracle=Randomized, Budget=350, Model=Flan-T5-XL2026.05 | 68.25 | |
| HeapSortOracle=Bidirectional, Budget=500, Model=Flan-T5-XL2026.05 | 68.21 | |
| Mohajer + BubbleOracle=Randomized, Budget=300, Model=Flan-T5-XL2026.05 | 68.14 | |
| Mohajer + BubbleOracle=Randomized, Budget=400, Model=Flan-T5-XL2026.05 | 68.08 | |
| Mohajer + BubbleOracle=Randomized, Budget=450, Model=Flan-T5-XL2026.05 | 68.08 | |
| Mohajer + BubbleOracle=Randomized, Budget=500, Model=Flan-T5-XL2026.05 | 68.08 | |
| MohajerOracle=Randomized, Budget=250, Model=Flan-T5-XL2026.05 | 68 | |
| MohajerOracle=Randomized, Budget=300, Model=Flan-T5-XL2026.05 | 68 | |
| MohajerOracle=Randomized, Budget=350, Model=Flan-T5-XL2026.05 | 68 | |
| MohajerOracle=Randomized, Budget=400, Model=Flan-T5-XL2026.05 | 68 | |
| MohajerOracle=Randomized, Budget=450, Model=Flan-T5-XL2026.05 | 68 | |
| MohajerOracle=Randomized, Budget=500, Model=Flan-T5-XL2026.05 | 68 | |
| Mohajer + BubbleOracle=Randomized, Budget=200, Model=Flan-T5-XL2026.05 | 67.66 | |
| MohajerOracle=Randomized, Budget=200, Model=Flan-T5-XL2026.05 | 67.66 | |
| Mohajer + BubbleOracle=Randomized, Budget=250, Model=Flan-T5-XL2026.05 | 67.59 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=450, Model=Flan-T5-XL2026.05 | 67.02 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=500, Model=Flan-T5-XL2026.05 | 67.02 | |
| MohajerOracle=Bidirectional, Budget=450, Model=Flan-T5-XL2026.05 | 66.96 | |
| MohajerOracle=Bidirectional, Budget=500, Model=Flan-T5-XL2026.05 | 66.96 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=400, Model=Flan-T5-XL2026.05 | 66.83 | |
| MohajerOracle=Bidirectional, Budget=400, Model=Flan-T5-XL2026.05 | 66.81 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=350, Model=Flan-T5-XL2026.05 | 66.28 | |
| MohajerOracle=Bidirectional, Budget=350, Model=Flan-T5-XL2026.05 | 66.28 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=300, Model=Flan-T5-XL2026.05 | 66.09 | |
| MohajerOracle=Bidirectional, Budget=300, Model=Flan-T5-XL2026.05 | 66.09 | |
| Mohajer + BubbleOracle=Randomized, Budget=150, Model=Flan-T5-XL2026.05 | 65.84 | |
| MohajerOracle=Randomized, Budget=150, Model=Flan-T5-XL2026.05 | 65.84 | |
| HeapSortOracle=Randomized, Budget=250, Model=Flan-T5-XL2026.05 | 65.8 | |
| BubbleSortOracle=Randomized, Budget=500, Model=Flan-T5-XL2026.05 | 65.42 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=250, Model=Flan-T5-XL2026.05 | 64.8 | |
| MohajerOracle=Bidirectional, Budget=250, Model=Flan-T5-XL2026.05 | 64.8 | |
| BubbleSortOracle=Randomized, Budget=400, Model=Flan-T5-XL2026.05 | 64.04 | |
| BubbleSortOracle=Randomized, Budget=450, Model=Flan-T5-XL2026.05 | 64 | |
| QuickSortOracle=Randomized, Budget=500, Model=Flan-T5-XL2026.05 | 63.76 | |
| HeapSortOracle=Bidirectional, Budget=450, Model=Flan-T5-XL2026.05 | 62.81 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=200, Model=Flan-T5-XL2026.05 | 62.34 | |
| MohajerOracle=Bidirectional, Budget=200, Model=Flan-T5-XL2026.05 | 62.34 | |
| BubbleSortOracle=Randomized, Budget=350, Model=Flan-T5-XL2026.05 | 62.03 | |
| BubbleSortOracle=Randomized, Budget=300, Model=Flan-T5-XL2026.05 | 61.95 | |
| QuickSortOracle=Randomized, Budget=450, Model=Flan-T5-XL2026.05 | 61.87 | |
| Mohajer + BubbleOracle=Randomized, Budget=100, Model=Flan-T5-XL2026.05 | 61.36 | |
| MohajerOracle=Randomized, Budget=100, Model=Flan-T5-XL2026.05 | 61.36 | |
| PAC + BubbleOracle=Bidirectional, Budget=400, Model=Flan-T5-XL2026.05 | 60.61 | |
| PAC + BubbleOracle=Bidirectional, Budget=450, Model=Flan-T5-XL2026.05 | 60.61 | |
| PAC + BubbleOracle=Bidirectional, Budget=500, Model=Flan-T5-XL2026.05 | 60.61 | |
| PAC + BubbleOracle=Bidirectional, Budget=350, Model=Flan-T5-XL2026.05 | 60.59 | |
| BubbleSortOracle=Bidirectional, Budget=500, Model=Flan-T5-XL2026.05 | 60.51 | |
| BubbleSortOracle=Bidirectional, Budget=450, Model=Flan-T5-XL2026.05 | 60.3 | |
| BubbleSortOracle=Bidirectional, Budget=400, Model=Flan-T5-XL2026.05 | 60.25 | |
| PAC + BubbleOracle=Randomized, Budget=200, Model=Flan-T5-XL2026.05 | 60.01 | |
| PAC + BubbleOracle=Randomized, Budget=250, Model=Flan-T5-XL2026.05 | 60.01 | |
| PAC + BubbleOracle=Randomized, Budget=300, Model=Flan-T5-XL2026.05 | 60.01 | |
| PAC + BubbleOracle=Randomized, Budget=350, Model=Flan-T5-XL2026.05 | 60.01 | |
| PAC + BubbleOracle=Randomized, Budget=400, Model=Flan-T5-XL2026.05 | 60.01 | |
| PAC + BubbleOracle=Randomized, Budget=450, Model=Flan-T5-XL2026.05 | 60.01 | |
| PAC + BubbleOracle=Randomized, Budget=500, Model=Flan-T5-XL2026.05 | 60.01 | |
| BubbleSortOracle=Randomized, Budget=200, Model=Flan-T5-XL2026.05 | 59.82 | |
| QuickSortOracle=Randomized, Budget=400, Model=Flan-T5-XL2026.05 | 59.81 | |
| BubbleSortOracle=Randomized, Budget=250, Model=Flan-T5-XL2026.05 | 59.68 | |
| QuickSortOracle=Randomized, Budget=350, Model=Flan-T5-XL2026.05 | 58.95 | |
| QuickSortOracle=Randomized, Budget=300, Model=Flan-T5-XL2026.05 | 57.56 | |
| PAC + BubbleOracle=Bidirectional, Budget=300, Model=Flan-T5-XL2026.05 | 57.52 | |
| PAC + BubbleOracle=Randomized, Budget=150, Model=Flan-T5-XL2026.05 | 57.02 | |
| BubbleSortOracle=Bidirectional, Budget=350, Model=Flan-T5-XL2026.05 | 56.98 | |
| QuickSortOracle=Bidirectional, Budget=500, Model=Flan-T5-XL2026.05 | 56.68 | |
| QuickSortOracle=Bidirectional, Budget=450, Model=Flan-T5-XL2026.05 | 56.59 | |
| BubbleSortOracle=Bidirectional, Budget=200, Model=Flan-T5-XL2026.05 | 56.43 | |
| BubbleSortOracle=Bidirectional, Budget=250, Model=Flan-T5-XL2026.05 | 56.43 | |
| BubbleSortOracle=Bidirectional, Budget=300, Model=Flan-T5-XL2026.05 | 56.42 | |
| QuickSortOracle=Bidirectional, Budget=400, Model=Flan-T5-XL2026.05 | 56.4 | |
| QuickSortOracle=Bidirectional, Budget=250, Model=Flan-T5-XL2026.05 | 56.2 | |
| QuickSortOracle=Bidirectional, Budget=300, Model=Flan-T5-XL2026.05 | 56.2 | |
| QuickSortOracle=Bidirectional, Budget=350, Model=Flan-T5-XL2026.05 | 56.2 | |
| QuickSortOracle=Randomized, Budget=250, Model=Flan-T5-XL2026.05 | 56.2 | |
| BubbleSortOracle=Randomized, Budget=150, Model=Flan-T5-XL2026.05 | 56.1 | |
| QuickSortOracle=Bidirectional, Budget=100, Model=Flan-T5-XL2026.05 | 55.93 | |
| BubbleSortOracle=Randomized, Budget=100, Model=Flan-T5-XL2026.05 | 55.9 | |
| QuickSortOracle=Bidirectional, Budget=150, Model=Flan-T5-XL2026.05 | 55.89 | |
| QuickSortOracle=Bidirectional, Budget=200, Model=Flan-T5-XL2026.05 | 55.87 | |
| QuickSortOracle=Randomized, Budget=200, Model=Flan-T5-XL2026.05 | 55.26 | |
| QuickSortOracle=Randomized, Budget=150, Model=Flan-T5-XL2026.05 | 54.71 | |
| QuickSortOracle=Randomized, Budget=100, Model=Flan-T5-XL2026.05 | 54.49 | |
| HeapSortOracle=Bidirectional, Budget=400, Model=Flan-T5-XL2026.05 | 54.29 | |
| HeapSortOracle=Randomized, Budget=200, Model=Flan-T5-XL2026.05 | 50.17 | |
| BubbleSortOracle=Bidirectional, Budget=100, Model=Flan-T5-XL2026.05 | 49.27 | |
| BubbleSortOracle=Bidirectional, Budget=150, Model=Flan-T5-XL2026.05 | 49.27 | |
| PAC + BubbleOracle=Bidirectional, Budget=100, Model=Flan-T5-XL2026.05 | 49.27 | |
| PAC + BubbleOracle=Bidirectional, Budget=150, Model=Flan-T5-XL2026.05 | 49.27 | |
| PAC + BubbleOracle=Bidirectional, Budget=200, Model=Flan-T5-XL2026.05 | 49.27 | |
| PAC + BubbleOracle=Bidirectional, Budget=250, Model=Flan-T5-XL2026.05 | 49.27 | |
| PAC + BubbleOracle=Randomized, Budget=100, Model=Flan-T5-XL2026.05 | 49.27 | |
| HeapSortOracle=Bidirectional, Budget=350, Model=Flan-T5-XL2026.05 | 41.84 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=100, Model=Flan-T5-XL2026.05 | 30.12 | |
| Mohajer + BubbleOracle=Bidirectional, Budget=150, Model=Flan-T5-XL2026.05 | 30.12 | |
| MohajerOracle=Bidirectional, Budget=100, Model=Flan-T5-XL2026.05 | 30.12 |