Summarization on CNN Daily Mail
47.97ROUGE-1PEGASUS-2B (calibrated)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| PEGASUS-2B (calibrated)#params=2B, beam size=10, training=SLiC2022.09 | 47.97 | 24.18 | 44.88 | |
| BRIO#params=340M2022.09 | 47.78 | 23.55 | 44.57 | |
| BART#Bits (E-W-A)=32-32-32, Size (MB)=532.0, FLOPS=1x2023.06 | 44.9 | 22.25 | 42.09 | |
| DQ-BART#Bits (E-W-A)=8-8-8, Size (MB)=138.12023.06 | 44.66 | 21.92 | 41.86 | |
| HAT-BART2021.12 | 44.48 | 21.31 | 41.52 | |
| MatchSum2026.02 | 44.41 | 20.86 | 40.55 | |
| PEGASUS-2B (fine-tuned)#params=2B, beam size=52022.09 | 44.31 | 21.91 | 41.41 | |
| AFLORAModel=BART-Large, Low-rank val.=16, #Params.=5.10M2024.03 | 44.31 | 21.32 | — | |
| PEGASUS2026.02 | 44.17 | 21.47 | 41.11 | |
| BART2026.02 | 44.16 | 21.28 | 40.09 | |
| GloSA-sum2026.02 | 44.05 | 21.22 | 41.06 | |
| LoRAModel=BART-Large, Low-rank val.=16, #Params.=8.65M2024.03 | 43.96 | 21.06 | — | |
| LongT5Model Size=xl, Input Length=4k, Attention=TGlobal2021.12 | 43.94 | 21.4 | 41.28 | |
| BigBird2026.02 | 43.83 | 21.11 | 40.74 | |
| TBT#Bits (E-W-A)=2-2-8, Size (MB)=39.6, FLOPS=0.25x2023.06 | 43.46 | 20.52 | 40.58 | |
| T5.1.1 + PSGModel Size=large, Input Length=4k2021.12 | 43.41 | 20.99 | 40.77 | |
| UNILMSummarization Type=Abstractive2019.05 | 43.33 | 20.21 | 40.51 | |
| Best ExtractiveSummarization Type=Extractive2019.05 | 43.25 | 20.24 | 39.63 | |
| Baseline (TWN)#Bits (E-W-A)=2-2-8, Size (MB)=39.6, FLOPS=0.25x2023.06 | 42.99 | 20.05 | 40.18 | |
| DQ-BART#Bits (E-W-A)=2-2-8, Size (MB)=39.6, FLOPS=0.25x2023.06 | 42.94 | 20.07 | 40.13 | |
| T5.1.1 + PSGModel Size=large, Input Length=2k2021.12 | 42.69 | 20.4 | 40.06 | |
| TBT#Bits (E-W-A)=1-1-8, Size (MB)=23.2, FLOPS=0.125x2023.06 | 42.66 | 19.72 | 39.8 | |
| T5.1.1 + PSGModel Size=large, Input Length=1k2021.12 | 42.62 | 20.46 | 40.02 | |
| T5.1.1Model Size=large, Input Length=1k2021.12 | 42.6 | 20.41 | 40.03 | |
| T5.1.1Model Size=large, Input Length=2k2021.12 | 42.55 | 20.25 | 39.99 | |
| LongT5Model Size=large, Input Length=4k, Attention=TGlobal2021.12 | 42.49 | 20.51 | 40.18 | |
| T5.1.1Model Size=large, Input Length=4k2021.12 | 42.27 | 19.93 | 39.72 | |
| LongT5Model Size=base, Input Length=4k, Attention=TGlobal2021.12 | 42.15 | 20.11 | 39.6 | |
| BERTSum2026.02 | 41.63 | 19.44 | 40.13 | |
| S2S-ELMoSummarization Type=Abstractive2019.05 | 41.56 | 18.94 | 38.47 | |
| BlockPruningSize (MB)=232023.06 | 41.4 | 18.7 | 38.4 | |
| Bottom-UpSummarization Type=Abstractive2019.05 | 41.22 | 18.68 | 38.34 | |
| TBT#Bits (E-W-A)=2-2-2, Size (MB)=39.6, FLOPS=0.0625x2023.06 | 41.03 | 18.18 | 38.3 | |
| LoKrBackbone=BART-large, Trainable Parameters=1.089M2024.10 | 40.83 | 19.1 | 38.75 | |
| RoCoFT-RowBackbone=BART-large, Trainable Parameters=0.597M2024.10 | 40.83 | 19.12 | 39.55 | |
| LoRABackbone=BART-large, Trainable Parameters=1.242M2024.10 | 40.74 | 19.1 | 39.24 | |
| BinaryBert*#Bits (E-W-A)=1-1-8, Size (MB)=23.2, FLOPS=0.125x2023.06 | 40.66 | 18.52 | 28.36 | |
| AdaLoRABackbone=BART-large, Trainable Parameters=1.663M2024.10 | 40.53 | 18.24 | 39.63 | |
| LEAD-3Summarization Type=Extractive2019.05 | 40.42 | 17.62 | 36.67 | |
| RoCoFT-ColumnBackbone=BART-large, Trainable Parameters=0.597M2024.10 | 40.18 | 19.1 | 39.21 | |
| LoHaBackbone=BART-large, Trainable Parameters=1.285M2024.10 | 39.98 | 18.84 | 38.01 | |
| Lead-32026.02 | 39.94 | 17.46 | 36.06 | |
| BitfitBackbone=BART-large, Trainable Parameters=0.672M2024.10 | 39.93 | 18.12 | 38.85 | |
| PGNetSummarization Type=Abstractive2019.05 | 39.53 | 17.28 | 37.98 | |
| FTBackbone=BART-large, Trainable Parameters=460M2024.10 | 39.16 | 18.92 | 37.04 | |
| Prefix-tuningBackbone=BART-large, Trainable Parameters=2.983M2024.10 | 38.32 | 17.72 | 35.76 | |
| Prompt tuningBackbone=BART-large, Trainable Parameters=0.755M2024.10 | 37.42 | 17.43 | 34.92 | |
| TBT#Bits (E-W-A)=1-1-1, Size (MB)=23.2, FLOPS=0.0156x2023.06 | 35.56 | 11.71 | 33.23 | |
| TextRank2026.02 | 33.1 | 12.2 | 29.7 | |
| BridgeBackbone=DS-Qwen-7B, Generation Width=82025.10 | 24.55 | — | — | |
| DS-Qwen-7BBackbone=DS-Qwen-7B2025.10 | 24.19 | — | — | |
| BridgeBackbone=DS-Qwen-1.5B, Generation Width=82025.10 | 24.07 | — | — | |
| DS-Llama-8BBackbone=DS-Llama-8B2025.10 | 23.84 | — | — | |
| P-MatchBackbone=DS-Qwen-7B2025.10 | 23.76 | — | — | |
| RLVRBackbone=DS-Qwen-7B2025.10 | 23.52 | — | — | |
| P-MatchBackbone=DS-Llama-8B2025.10 | 23.01 | — | — | |
| RLVRBackbone=DS-Qwen-1.5B2025.10 | 22.79 | — | — | |
| P-MatchBackbone=DS-Qwen-1.5B2025.10 | 22.78 | — | — | |
| BridgeBackbone=DS-Llama-8B, Generation Width=82025.10 | 22.64 | — | — | |
| DS-Qwen-1.5BBackbone=DS-Qwen-1.5B2025.10 | 22.11 | — | — | |
| BiBert*#Bits (E-W-A)=1-1-1, Size (MB)=23.2, FLOPS=0.0156x2023.06 | 14.22 | 0.13 | 10.06 | |
| Baseline (TWN)#Bits (E-W-A)=2-2-2, Size (MB)=39.6, FLOPS=0.0625x2023.06 | 12.92 | 0.32 | 12.42 | |
| TernaryBert*#Bits (E-W-A)=2-2-2, Size (MB)=39.6, FLOPS=0.0625x2023.06 | 10.95 | 0.52 | 8.56 | |
| BinaryBert*#Bits (E-W-A)=1-1-1, Size (MB)=23.2, FLOPS=0.0156x2023.06 | 9.8 | 0.15 | 8.62 | |
| Baseline (BWN)#Bits (E-W-A)=1-1-8, Size (MB)=23.2, FLOPS=0.125x2023.06 | 2.78 | 0.08 | 2.48 | |
| Baseline (BWN)#Bits (E-W-A)=1-1-1, Size (MB)=23.2, FLOPS=0.0156x2023.06 | 2.78 | 0.08 | 2.48 | |
| RLVRBackbone=DS-Llama-8B2025.10 | 1.65 | — | — |