Summarization on CNN/DM (R1, R2, RL, BS, MS, EM, CM, SM, UE, Fe, Fi, SC)
51.85ROUGE-1Oracle
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OracleSetting=epsilon, Reranker=Oracle2026.05 | 51.85 | 28.45 | 45.15 | 71.35 | 61.29 | 23.82 | -0.47 | 35.02 | 95.2 | 99.8 | 84.22 | 99.89 | |
| OracleSetting=beam-sim, Reranker=Oracle2026.05 | 46.68 | 27.01 | 41.8 | 69.9 | 59.54 | 39.8 | -0.47 | 23.46 | 96.12 | 99.86 | 92.69 | 98.62 | |
| OracleSetting=llm, Reranker=Oracle2026.05 | 40.88 | 18.83 | 34.05 | 67.03 | — | 14.81 | -0.45 | 49.41 | 95.63 | 99.81 | 55.71 | 99.95 | |
| FIZZ-0.75Setting=epsilon, Reranker=FIZZ-0.752026.05 | 40.67 | 17.87 | 34.36 | 65.94 | 58.24 | 7.68 | -4.46 | 2.44 | 84.16 | 98.45 | — | 99.7 | |
| FENICE-0.75Setting=epsilon, Reranker=FENICE-0.752026.05 | 40.6 | 17.47 | 34.11 | 65.86 | 58.25 | 10.44 | -3.82 | 2.36 | 83.34 | — | 52.44 | 99.7 | |
| FIZZ-0.0Setting=epsilon, Reranker=FIZZ-0.02026.05 | 40.54 | 17.81 | 34.23 | 65.91 | 58.17 | 5.18 | -5.59 | 2.39 | 84.65 | 98.39 | — | 99.72 | |
| FENICE-0.0Setting=epsilon, Reranker=FENICE-0.02026.05 | 40.52 | 17.46 | 34.06 | 66.02 | 58.19 | 5.96 | -5.21 | 2.14 | 85.18 | — | 55.83 | 99.71 | |
| MBR-1.0Setting=epsilon, Reranker=MBR-1.02026.05 | 40.36 | 17.23 | 33.87 | 65.6 | 58.19 | 11.29 | -3.78 | — | 81.41 | 97.35 | 47.26 | 99.69 | |
| BaselineSetting=epsilon, Reranker=Baseline2026.05 | 39.96 | 16.88 | 33.38 | 65.52 | 58.03 | 4.46 | -6.62 | 1.63 | 81.37 | 95.52 | 39.36 | 99.74 | |
| MBR-1.0Setting=beam-sim, Reranker=MBR-1.02026.05 | 36.56 | 16.65 | 31.14 | 64.88 | 56.87 | 12.03 | -4.44 | — | 89.67 | 97.32 | 60.6 | 97.73 | |
| FENICE-0.75Setting=beam-sim, Reranker=FENICE-0.752026.05 | 36.5 | 16.63 | 31.23 | 64.97 | 56.9 | 13.37 | -4.54 | 6.6 | 90.14 | — | 65.85 | 97.71 | |
| BaselineSetting=beam-sim, Reranker=Baseline2026.05 | 36.37 | 17.34 | 31.64 | 64.78 | 56.79 | 12.26 | -5.67 | 2.24 | 89.86 | 97.02 | 68.44 | 97.63 | |
| FIZZ-0.75Setting=beam-sim, Reranker=FIZZ-0.752026.05 | 36.14 | 16.65 | 31.14 | 64.68 | 56.79 | 13.19 | -4.99 | 4.59 | 89.48 | 97.8 | — | 97.65 | |
| FENICE-0.0Setting=beam-sim, Reranker=FENICE-0.02026.05 | 35.69 | 15.94 | 30.67 | 64.61 | 56.74 | 14.56 | -5.49 | -0.13 | 89.54 | — | 68.89 | 97.65 | |
| FIZZ-0.0Setting=beam-sim, Reranker=FIZZ-0.02026.05 | 35.6 | 16.27 | 30.78 | 64.37 | 56.65 | 12.8 | -5.91 | 0.45 | 88.94 | 97.66 | — | 97.62 | |
| FIZZ-0.75Setting=llm, Reranker=FIZZ-0.752026.05 | 35.36 | 13.98 | 28.8 | 64.34 | 56.68 | 5.34 | -1.95 | 25.08 | 92.7 | 98.9 | — | 99.9 | |
| FENICE-0.75Setting=llm, Reranker=FENICE-0.752026.05 | 35.31 | 14.11 | 28.71 | 64.38 | 56.66 | 5.95 | -1.6 | 31.14 | 92.91 | — | 28.5 | 99.91 | |
| FIZZ-0.0Setting=llm, Reranker=FIZZ-0.02026.05 | 35.26 | 13.8 | 28.74 | 64.23 | 56.66 | 5.13 | -2.27 | 20.92 | 92.55 | 98.89 | — | 99.9 | |
| FENICE-0.0Setting=llm, Reranker=FENICE-0.02026.05 | 35.15 | 13.77 | 28.66 | 64.2 | 56.64 | 5.53 | -2.05 | 21.19 | 92.63 | — | 31.12 | 99.9 | |
| MBR-1.0Setting=llm, Reranker=MBR-1.02026.05 | 35.15 | 14.07 | 28.52 | 64.36 | 56.63 | 6.02 | -1.51 | — | 92.99 | 98.5 | 24.73 | 99.92 | |
| BaselineSetting=llm, Reranker=Baseline2026.05 | 34.83 | 13.51 | 28.34 | 64.04 | 56.56 | 4.8 | -2.12 | 21.7 | 92.58 | 98.43 | 24.98 | 99.91 |