Abstractive Summarization on CNN/DailyMail (Hamming, F1, Align Score)
1.597Hamming DistanceBayes (ours)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Bayes (ours)Model=Gemma-3-12B, Decoding Strategy=Bayes (ours)2026.01 | 1.597 | 0.231 | 0.888 | |
| Bayes (ours)Model=Gemma-3-4B, Decoding Strategy=Bayes (ours)2026.01 | 1.603 | 0.224 | 0.877 | |
| ℓSampleModel=Gemma-3-12B, Decoding Strategy=ℓSample2026.01 | 1.846 | 0.203 | 0.869 | |
| Bayes (ours)Model=Qwen3-4B, Decoding Strategy=Bayes (ours)2026.01 | 1.851 | 0.222 | 0.87 | |
| ℓMAPModel=Gemma-3-12B, Decoding Strategy=ℓMAP2026.01 | 1.912 | 0.204 | 0.843 | |
| ℓSampleModel=Gemma-3-4B, Decoding Strategy=ℓSample2026.01 | 1.93 | 0.211 | 0.85 | |
| Bayes (ours)Model=Qwen3-30B-A3B, Decoding Strategy=Bayes (ours)2026.01 | 2.008 | 0.197 | 0.863 | |
| BeamModel=Gemma-3-12B, Decoding Strategy=Beam2026.01 | 2.051 | 0.203 | 0.843 | |
| ℓMAPModel=Gemma-3-4B, Decoding Strategy=ℓMAP2026.01 | 2.06 | 0.185 | 0.832 | |
| Self-ConsistencyModel=Gemma-3-12B, Decoding Strategy=Self-Consistency2026.01 | 2.074 | 0.214 | 0.86 | |
| ℓSampleModel=Qwen3-4B, Decoding Strategy=ℓSample2026.01 | 2.074 | 0.214 | 0.854 | |
| BeamModel=Gemma-3-4B, Decoding Strategy=Beam2026.01 | 2.176 | 0.192 | 0.831 | |
| ℓMAPModel=Qwen3-4B, Decoding Strategy=ℓMAP2026.01 | 2.212 | 0.178 | 0.846 | |
| Self-ConsistencyModel=Gemma-3-4B, Decoding Strategy=Self-Consistency2026.01 | 2.219 | 0.183 | 0.852 | |
| BeamModel=Qwen3-4B, Decoding Strategy=Beam2026.01 | 2.238 | 0.208 | 0.849 | |
| Self-ConsistencyModel=Qwen3-4B, Decoding Strategy=Self-Consistency2026.01 | 2.24 | 0.21 | 0.842 | |
| ℓSampleModel=Qwen3-30B-A3B, Decoding Strategy=ℓSample2026.01 | 2.249 | 0.192 | 0.841 | |
| ℓMAPModel=Qwen3-30B-A3B, Decoding Strategy=ℓMAP2026.01 | 2.334 | 0.189 | 0.837 | |
| BeamModel=Qwen3-30B-A3B, Decoding Strategy=Beam2026.01 | 2.422 | 0.183 | 0.841 | |
| Self-ConsistencyModel=Qwen3-30B-A3B, Decoding Strategy=Self-Consistency2026.01 | 2.427 | 0.184 | 0.807 |