Reranking on CNN DailyMail
52.43R-1Oracle
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| OracleGenerator Backbone=BART-Large2026.03 | 52.43 | 28.73 | 49.42 | 89.3 | |
| OracleGenerator Backbone=T5-base2026.03 | 48.21 | 24.95 | 45.29 | 88.32 | |
| OracleGenerator Backbone=Llama-3.1-8B, Number of shots=3-shot2026.03 | 46.35 | 19.72 | 42.75 | 88.49 | |
| RankGPT(Qwen2.5-32B)Generator Backbone=BART-Large2026.03 | 44.54 | 21.17 | 41.33 | 88.04 | |
| FlashEvaluatorGenerator Backbone=BART-Large, Per-epoch training time=∼25min2026.03 | 44.52 | 21.35 | 41.48 | 88.2 | |
| SimCLSGenerator Backbone=BART-Large, Per-epoch training time=∼50min2026.03 | 44.46 | 21.12 | 41.29 | 88.03 | |
| Base (Beam 1)Generator Backbone=BART-Large2026.03 | 43.63 | 20.66 | 40.47 | 87.88 | |
| SimCLSGenerator Backbone=T5-base, Per-epoch training time=∼50min2026.03 | 42.12 | 19.33 | 39.14 | 87.51 | |
| SimCLSGenerator Backbone=Llama-3.1-8B, Number of shots=3-shot, Per-epoch training time=∼50min2026.03 | 41.29 | 15.53 | 37.48 | 87.6 | |
| FlashEvaluatorGenerator Backbone=T5-base, Per-epoch training time=∼25min2026.03 | 41.28 | 18.71 | 38.35 | 87.42 | |
| RankGPT(Qwen2.5-32B)Generator Backbone=T5-base2026.03 | 41.09 | 18.35 | 37.99 | 87.32 | |
| Base (Beam 1)Generator Backbone=T5-base2026.03 | 40.81 | 18.37 | 37.81 | 87.23 | |
| FlashEvaluatorGenerator Backbone=Llama-3.1-8B, Number of shots=3-shot, Per-epoch training time=∼25min2026.03 | 40.46 | 14.65 | 36.7 | 87.77 | |
| RankGPT(Qwen2.5-32B)Generator Backbone=Llama-3.1-8B, Number of shots=3-shot2026.03 | 39 | 13.82 | 35.33 | 87.51 | |
| Base (Beam 1)Generator Backbone=Llama-3.1-8B, Number of shots=3-shot2026.03 | 38.78 | 13.44 | 35.08 | 87.53 |