Summarization on TrueBrief (test)
52ROUGE-1SFT
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| SFTModel Size=3B, Backbone=Qwen2.52025.12 | 52 | 26 | 35 | 71 | 3.52 | 4.22 | 4.72 | 4.59 | 93 | 82 | |
| SFTModel Size=7B, Backbone=Qwen2.52025.12 | 52 | 26 | 35 | 71 | 3.79 | 4.38 | 4.83 | 4.72 | 96 | 85 | |
| DPOModel Size=7B, Backbone=Qwen2.52025.12 | 52 | 26 | 35 | 71 | 3.8 | 4.39 | 4.85 | 4.73 | 96 | 86 | |
| DPOModel Size=3B, Backbone=Qwen2.52025.12 | 51 | 25 | 34 | 70 | 3.43 | 4.09 | 4.71 | 4.48 | 93 | 82 | |
| Add-DPOModel Size=3B, Backbone=Qwen2.52025.12 | 51 | 24 | 33 | 70 | 3.46 | 4.16 | 4.73 | 4.54 | 93 | 81 | |
| PL-DPOModel Size=3B, Backbone=Qwen2.52025.12 | 51 | 26 | 34 | 70 | 3.51 | 4.05 | 4.68 | 4.51 | 92 | 81 | |
| BaselineModel Size=3B, Backbone=Qwen2.52025.12 | 47 | 21 | 31 | 68 | 3.3 | 4.23 | 4.68 | 4.53 | 91 | 79 | |
| BaselineModel Size=7B, Backbone=Qwen2.52025.12 | 47 | 20 | 31 | 68 | 3.3 | 4.24 | 4.73 | 4.6 | 95 | 80 | |
| DPOModel Size=1.5B, Backbone=Qwen2.52025.12 | 46 | 20 | 30 | 68 | 3.2 | 3.81 | 4.44 | 4.23 | 86 | 75 | |
| Add-DPOModel Size=1.5B, Backbone=Qwen2.52025.12 | 46 | 21 | 30 | 68 | 3.15 | 3.71 | 4.4 | 4.17 | 84 | 73 | |
| PL-DPOModel Size=1.5B, Backbone=Qwen2.52025.12 | 46 | 20 | 30 | 68 | 3.22 | 3.67 | 4.37 | 4.16 | 85 | 74 | |
| Sep-DPOModel Size=0.5B, Backbone=Qwen2.52025.12 | 45 | 19 | 28 | 67 | 2.69 | 3.21 | 3.85 | 3.52 | 74 | 64 | |
| PL-DPOModel Size=0.5B, Backbone=Qwen2.52025.12 | 45 | 20 | 29 | 67 | 2.54 | 3.13 | 3.85 | 3.5 | 73 | 62 | |
| SFTModel Size=1.5B, Backbone=Qwen2.52025.12 | 45 | 20 | 29 | 67 | 3.15 | 3.63 | 4.23 | 4.02 | 84 | 73 | |
| SFTModel Size=0.5B, Backbone=Qwen2.52025.12 | 44 | 19 | 29 | 66 | 2.54 | 3.15 | 3.91 | 3.52 | 73 | 62 | |
| DPOModel Size=0.5B, Backbone=Qwen2.52025.12 | 44 | 19 | 28 | 66 | 2.66 | 3.19 | 3.91 | 3.56 | 77 | 65 | |
| Add-DPOModel Size=0.5B, Backbone=Qwen2.52025.12 | 43 | 17 | 27 | 65 | 2.54 | 3.12 | 3.83 | 3.47 | 74 | 62 | |
| BaselineModel Size=1.5B, Backbone=Qwen2.52025.12 | 41 | 16 | 25 | 65 | 3.22 | 3.71 | 4.45 | 4.3 | 81 | 73 | |
| BaselineModel Size=0.5B, Backbone=Qwen2.52025.12 | 40 | 16 | 25 | 64 | 2.57 | 3.18 | 3.92 | 3.6 | 72 | 62 |