Summarization on CNN/DailyMail (Hamming, F1, Align Score)
-0.276Hamming ScoreKLE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| KLEBackbone=Qwen3-4B2026.01 | -0.276 | 6.6 | 1.4 | — | — | — | |
| SEBackbone=Qwen3-30B-A3B2026.01 | -0.269 | 13.1 | -14.1 | — | — | — | |
| SEBackbone=Qwen3-4B2026.01 | -0.242 | -0.7 | 2.8 | — | — | — | |
| SARBackbone=Gemma-3-4B2026.01 | -0.241 | 13.5 | -2 | — | — | — | |
| MSPBackbone=Qwen3-4B2026.01 | -0.217 | 21.1 | 5.8 | — | — | — | |
| P(True)Backbone=Gemma-3-4B2026.01 | -0.165 | -6.9 | -9.7 | — | — | — | |
| MSPBackbone=Gemma-3-12B2026.01 | -0.162 | 22.1 | -19.5 | — | — | — | |
| SARBackbone=Qwen3-4B2026.01 | -0.161 | 17.6 | 9.7 | — | — | — | |
| COCOABackbone=Qwen3-4B2026.01 | -0.158 | 28.4 | 9.5 | — | — | — | |
| SARBackbone=Gemma-3-12B2026.01 | -0.155 | 24.4 | -12.2 | — | — | — | |
| COCOABackbone=Gemma-3-12B2026.01 | -0.127 | 24 | -20 | — | — | — | |
| COCOABackbone=Gemma-3-4B2026.01 | -0.126 | 3.7 | -4.6 | — | — | — | |
| SEBackbone=Gemma-3-4B2026.01 | -0.104 | -3.9 | 1.7 | — | — | — | |
| KLEBackbone=Gemma-3-12B2026.01 | -0.1 | -1.4 | 5.6 | — | — | — | |
| MSPBackbone=Gemma-3-4B2026.01 | -0.063 | 3.1 | 1.4 | — | — | — | |
| MSPBackbone=Qwen3-30B-A3B2026.01 | -0.06 | 28.5 | -2.6 | — | — | — | |
| P(True)Backbone=Gemma-3-12B2026.01 | -0.051 | -23.6 | -5.9 | — | — | — | |
| KLEBackbone=Qwen3-30B-A3B2026.01 | -0.05 | 29.1 | -6.8 | — | — | — | |
| COCOABackbone=Qwen3-30B-A3B2026.01 | -0.034 | 26.8 | -9.7 | — | — | — | |
| SARBackbone=Qwen3-30B-A3B2026.01 | -0.026 | 27.5 | -8.8 | — | — | — | |
| KLEBackbone=Gemma-3-4B2026.01 | -0.024 | 1 | 6.7 | — | — | — | |
| P(True)Backbone=Qwen3-30B-A3B2026.01 | 0.009 | 11.9 | -13.9 | — | — | — | |
| SEBackbone=Gemma-3-12B2026.01 | 0.018 | 16.5 | 5.8 | — | — | — | |
| P(True)Backbone=Qwen3-4B2026.01 | 0.037 | -6.6 | 11.3 | — | — | — | |
| R_BayesBackbone=Qwen3-4B2026.01 | 0.055 | 11.5 | 27.8 | — | — | — | |
| R_BayesBackbone=Gemma-3-12B2026.01 | 0.144 | 25.7 | 32.1 | — | — | — | |
| R_BayesBackbone=Qwen3-30B-A3B2026.01 | 0.235 | 35.8 | 20.3 | — | — | — | |
| R_BayesBackbone=Gemma-3-4B2026.01 | 0.363 | 26.6 | 14.6 | — | — | — | |
| COCOAModel=Gemma-3-4B2026.01 | — | — | — | 0.449 | 0.536 | 0.513 | |
| COCOAModel=Gemma-3-12B2026.01 | — | — | — | 0.466 | 0.568 | 0.453 | |
| COCOAModel=Qwen3-4B2026.01 | — | — | — | 0.461 | 0.572 | 0.495 | |
| COCOAModel=Qwen3-30B-A3B2026.01 | — | — | — | 0.498 | 0.608 | 0.466 | |
| KLEModel=Gemma-3-4B2026.01 | — | — | — | 0.494 | 0.505 | 0.522 | |
| KLEModel=Gemma-3-12B2026.01 | — | — | — | 0.462 | 0.523 | 0.469 | |
| KLEModel=Qwen3-4B2026.01 | — | — | — | 0.428 | 0.537 | 0.479 | |
| KLEModel=Qwen3-30B-A3B2026.01 | — | — | — | 0.496 | 0.563 | 0.507 | |
| MSPModel=Gemma-3-4B2026.01 | — | — | — | 0.455 | 0.521 | 0.508 | |
| MSPModel=Gemma-3-12B2026.01 | — | — | — | 0.449 | 0.548 | 0.453 | |
| MSPModel=Qwen3-4B2026.01 | — | — | — | 0.464 | 0.563 | 0.5 | |
| MSPModel=Qwen3-30B-A3B2026.01 | — | — | — | 0.486 | 0.594 | 0.473 | |
| P(True)Model=Gemma-3-4B2026.01 | — | — | — | 0.468 | 0.504 | 0.459 | |
| P(True)Model=Gemma-3-12B2026.01 | — | — | — | 0.497 | 0.42 | 0.487 | |
| P(True)Model=Qwen3-4B2026.01 | — | — | — | 0.495 | 0.477 | 0.522 | |
| P(True)Model=Qwen3-30B-A3B2026.01 | — | — | — | 0.522 | 0.533 | 0.442 | |
| RBayesModel=Gemma-3-4B2026.01 | — | — | — | 0.6 | 0.584 | 0.58 | |
| RBayesModel=Gemma-3-12B2026.01 | — | — | — | 0.58 | 0.571 | 0.605 | |
| RBayesModel=Qwen3-4B2026.01 | — | — | — | 0.525 | 0.552 | 0.594 | |
| RBayesModel=Qwen3-30B-A3B2026.01 | — | — | — | 0.577 | 0.602 | 0.578 | |
| SARModel=Gemma-3-4B2026.01 | — | — | — | 0.449 | 0.561 | 0.514 | |
| SARModel=Gemma-3-12B2026.01 | — | — | — | 0.461 | 0.573 | 0.461 | |
| SARModel=Qwen3-4B2026.01 | — | — | — | 0.462 | 0.549 | 0.501 | |
| SARModel=Qwen3-30B-A3B2026.01 | — | — | — | 0.484 | 0.599 | 0.479 | |
| SEModel=Gemma-3-4B2026.01 | — | — | — | 0.468 | 0.496 | 0.521 | |
| SEModel=Gemma-3-12B2026.01 | — | — | — | 0.519 | 0.563 | 0.486 | |
| SEModel=Qwen3-4B2026.01 | — | — | — | 0.451 | 0.523 | 0.508 | |
| SEModel=Qwen3-30B-A3B2026.01 | — | — | — | 0.414 | 0.525 | 0.466 |