Abstractive Summarization on Multi-News
21.1ROUGE-2PRIMER
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PRIMER2021.12 | 21.1 | 49.9 | 25.9 | — | |
| LongT5Model Size=xl, Input Length=8k, Attention=TGlobal2021.12 | 19.43 | 48.17 | 24.94 | — | |
| LongT5Model Size=xl, Input Length=4k, Attention=TGlobal2021.12 | 19.3 | 48.15 | 24.76 | — | |
| PEGA-CONTBackbone=PEGASUS (large)2022.05 | 19.29 | 48.68 | 24.58 | — | |
| PEGASUS_LARGEPre-training strategy=mixed, stochastic2019.12 | 18.75 | 47.65 | 24.95 | — | |
| Previous SOTA2022.05 | 18.72 | 47.52 | 24.91 | — | |
| PEGASUSBackbone=large2022.05 | 18.72 | 47.52 | 24.91 | — | |
| T5.1.1 + PSGModel Size=large, Input Length=4k2021.12 | 18.6 | 47.48 | 24.31 | — | |
| LongT5Model Size=large, Input Length=8k, Attention=TGlobal2021.12 | 18.44 | 47.18 | 24.18 | — | |
| LongT5Model Size=large, Input Length=4k, Attention=TGlobal2021.12 | 18.21 | 46.99 | 24.08 | — | |
| T5.1.1Model Size=large, Input Length=4k2021.12 | 17.88 | 45.67 | 24.15 | — | |
| T5.1.1 + PSGModel Size=large, Input Length=2k2021.12 | 17.74 | 46.65 | 23.74 | — | |
| TG-MultiSum2021.12 | 17.55 | 47.1 | 20.73 | — | |
| LongT5Model Size=base, Input Length=4k, Attention=TGlobal2021.12 | 17.37 | 46.01 | 23.5 | — | |
| T5.1.1Model Size=large, Input Length=2k2021.12 | 17.26 | 44.95 | 23.74 | — | |
| OursFew-shot examples=1002021.10 | 16.76 | 46.01 | 22.91 | — | |
| T5-CONTBackbone=small2022.05 | 16.39 | 44.08 | 22.58 | — | |
| T5.1.1Model Size=large, Input Length=1k2021.12 | 16.26 | 43.69 | 23.03 | — | |
| UNISUMMFew-shot samples=1002022.11 | 15.86 | 45.91 | 22.24 | — | |
| T5.1.1 + PSGModel Size=large, Input Length=1k2021.12 | 15.85 | 44.43 | 22.41 | — | |
| T5-Naive CLBackbone=small, Loss=N-Pairs2022.05 | 15.83 | 43.04 | 22.03 | — | |
| MultiBARTFew-shot samples=1002022.11 | 15.78 | 45.71 | 22.21 | — | |
| OursFew-shot examples=102021.10 | 15.54 | 44.02 | 22.03 | — | |
| T5Backbone=small2022.05 | 15.34 | 42.36 | 21.91 | — | |
| T5-Dropout CLBackbone=small2022.05 | 15.32 | 42.43 | 21.95 | — | |
| UNISUMMFew-shot samples=102022.11 | 15.19 | 45.13 | 21.63 | — | |
| T5-Naive CLBackbone=small2022.05 | 15.18 | 42.2 | 21.78 | — | |
| T5-SSMBA CLBackbone=small2022.05 | 14.98 | 42.06 | 21.73 | — | |
| LEDFew-shot examples=1002021.10 | 14.85 | 44.45 | 21.16 | — | |
| BARTFew-shot examples=1002021.10 | 14.77 | 44.23 | 21.02 | — | |
| MultiBARTFew-shot samples=102022.11 | 14.55 | 42.31 | 21.53 | — | |
| BART-PTFew-shot samples=1002022.11 | 13.97 | 43.86 | 20.79 | — | |
| OursFew-shot examples=02021.10 | 13.91 | 39.09 | 19.19 | — | |
| BARTFew-shot examples=102021.10 | 13.74 | 42.3 | 19.71 | — | |
| PEGASUSFew-shot examples=1002021.10 | 13.5 | 42.99 | 21.1 | — | |
| PEGASUSBackbone=base2022.05 | 13.27 | 42.24 | 21.44 | — | |
| BART-FTFew-shot samples=1002022.11 | 13.27 | 42.65 | 20.69 | — | |
| PEGASUSFew-shot samples=1002022.11 | 12.78 | 42.36 | 20.56 | — | |
| LEDFew-shot examples=102021.10 | 12.48 | 38.86 | 18.82 | — | |
| PEGASUSFew-shot examples=102021.10 | 12.1 | 39.02 | 20.32 | — | |
| BART-PTFew-shot samples=102022.11 | 11.38 | 38.27 | 19.28 | — | |
| PEGASUSFew-shot samples=102022.11 | 11.15 | 39.12 | 19.44 | — | |
| GPT-3.5 (text-davinci-002)Few-shot samples=12022.11 | 11.01 | — | — | — | |
| PEGASUSFew-shot examples=02021.10 | 10.06 | 31.97 | 16.74 | — | |
| BART-FTFew-shot samples=102022.11 | 10.05 | 38.29 | 18.32 | — | |
| BARTFew-shot examples=02021.10 | 8.98 | 26.1 | 13.06 | — | |
| LEDFew-shot examples=02021.10 | 4.78 | 16.6 | 9.05 | — | |
| DCSbackbone=Llama-3-8B-Instruct, variant=with DCS2025.06 | — | — | — | 27.73 | |
| Infilmbackbone=Llama-3-8B-Instruct, variant=with Infilm2025.06 | — | — | — | 27.61 | |
| Llama-3-8B-Instructbackbone=Llama-3-8B-Instruct2025.06 | — | — | — | 27.78 | |
| LM-infinitebackbone=Llama-3-8B-Instruct, variant=with LM-infinite2025.06 | — | — | — | 27.66 | |
| Streamingbackbone=Llama-3-8B-Instruct, variant=with Streaming2025.06 | — | — | — | 27.73 |