Summarization on Summarize from Feedback (test)
23.73HelpfulnessSTaR
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| STaRBackbone=Llama 3B2025.12 | 23.73 | 69.49 | 27.15 | 71.33 | |
| STaRBackbone=Qwen 7B2025.12 | 22.61 | 67.32 | 19.98 | 71.32 | |
| PPSBackbone=Qwen 7B2025.12 | 22.61 | 71.37 | 21.12 | 73.04 | |
| PPSBackbone=Llama 3B2025.12 | 22.19 | 69.2 | 26.47 | 76.66 | |
| PPSBackbone=Llama 8B2025.12 | 20.93 | 72.5 | 24 | 74.22 | |
| PPSBackbone=Qwen 1B2025.12 | 20.64 | 66.14 | 24.75 | 76.26 | |
| STaRBackbone=Llama 8B2025.12 | 20.56 | 73.26 | 22.67 | 71.17 | |
| STaRBackbone=Qwen 1B2025.12 | 20.18 | 58.24 | 23.04 | 74.51 | |
| PPSBackbone=Qwen 3B2025.12 | 20.01 | 32.71 | 16.54 | 72.9 | |
| STaRBackbone=Qwen 3B2025.12 | 19.47 | 26.67 | 15.4 | 69.03 | |
| RSBackbone=Qwen 7B2025.12 | 18.76 | 10.35 | 19.03 | 5.26 | |
| RSBackbone=Llama 3B2025.12 | 18.61 | 14.21 | 18.08 | 7.44 | |
| BaseBackbone=Qwen 7B2025.12 | 17.85 | 8.54 | 19.22 | 5.14 | |
| RSBackbone=Qwen 3B2025.12 | 17.73 | 9.08 | 14.16 | 15.43 | |
| BaseBackbone=Qwen 3B2025.12 | 17.17 | 8.74 | 12.91 | 15.33 | |
| RSBackbone=Llama 8B2025.12 | 16.34 | 15.42 | 18.44 | 61.7 | |
| RSBackbone=Qwen 1B2025.12 | 14.56 | 36.93 | 9.3 | 20.11 | |
| BaseBackbone=Llama 3B2025.12 | 14.14 | 11.38 | 14.03 | 5.33 | |
| BaseBackbone=Llama 8B2025.12 | 13.96 | 9.89 | 17.59 | 11.41 | |
| BaseBackbone=Qwen 1B2025.12 | 13.86 | 22.83 | 8.08 | 17.27 |