Summarization on Gigaword (test)
20.7ROUGE-2Aghajanyan et al.
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Aghajanyan et al.status=previous state-of-the-art2022.05 | 20.7 | — | — | — | — | — | — | — | — | — | |
| UL20Bmode=supervised finetuning2022.05 | 20.7 | — | — | — | — | — | — | — | — | — | |
| BART+R3FSOTA status=New SOTA2020.08 | 20.69 | — | — | — | 40.45 | 36.56 | — | — | — | — | |
| CopyTranssearch_algorithm=Beam Search, normalization=SBWR, r=0.252019.11 | 20.47 | — | — | — | 39.08 | 36.68 | 61.51 | — | — | — | |
| ProphetNet2020.08 | 20.42 | — | — | — | 39.51 | 36.69 | — | — | — | — | |
| ProphetNetCorpus=160GB2020.01 | 20.42 | — | — | — | 39.51 | 36.69 | — | — | — | — | |
| CopyTranssearch_algorithm=Beam Search, normalization=BPNorm, c=0.552019.11 | 20.38 | — | — | — | 39.19 | 36.69 | 61.46 | — | — | — | |
| CopyTranssearch_algorithm=Beam Search2019.11 | 20.37 | — | — | — | 38.87 | 36.52 | 61.47 | — | — | — | |
| CopyTranssearch_algorithm=Best-first Search2019.11 | 20.28 | — | — | — | 39.07 | 36.49 | 61.27 | — | — | — | |
| CopyTranssearch_algorithm=Beam Search, normalization=LengthNorm2019.11 | 20.25 | — | — | — | 39.1 | 36.55 | 61.41 | — | — | — | |
| BART2020.08 | 20.09 | — | — | — | 39.29 | 35.65 | — | — | — | — | |
| GLMSentObjective=Sentence-level2021.03 | 20 | — | — | — | 38.9 | 36.3 | — | — | — | — | |
| GLM410MNumber of parameters=410M2021.03 | 20 | — | — | — | 38.9 | 36.2 | — | — | — | — | |
| PEGASUS_LARGE (C4)Corpus=750GB2020.01 | 19.96 | — | — | — | 38.75 | 36.14 | — | — | — | — | |
| PEGASUS2020.08 | 19.86 | — | — | — | 39.12 | 36.24 | — | — | — | — | |
| PEGASUS_LARGE (HugeNews)Corpus=3800GB2020.01 | 19.86 | — | — | — | 39.12 | 36.24 | — | — | — | — | |
| GLMLargeBackbone=Large2021.03 | 19.7 | — | — | — | 38.6 | 36 | — | — | — | — | |
| BISET2019.11 | 19.53 | — | — | — | 38.45 | 36.04 | 57.1 | — | — | — | |
| UniLMLargeBackbone=Large2021.03 | 19.5 | — | — | — | 38.5 | 35.8 | — | — | — | — | |
| GLMDOCObjective=Document-level2021.03 | 19.4 | — | — | — | 38.5 | 35.8 | — | — | — | — | |
| DASumModel=DASum2023.06 | 18.53 | — | — | — | 38.15 | 35.31 | 88.9 | 27.39 | 28.95 | 28.17 | |
| MASS2021.03 | 18.5 | — | — | — | 37.7 | 34.9 | — | — | — | — | |
| DASum w/o LdAblation=Without Ld2023.06 | 18.31 | — | — | — | 37.36 | 34.64 | 88.71 | 24.64 | 26.93 | 25.79 | |
| DASum w/o LcAblation=Without Lc2023.06 | 18.3 | — | — | — | 37.21 | 34.32 | 88.64 | 25.56 | 26.19 | 25.87 | |
| R3Sum2019.11 | 18.23 | — | — | — | 36.36 | 33.85 | 56.74 | — | — | — | |
| DASum w/o LeAblation=Without Le2023.06 | 18.17 | — | — | — | 36.71 | 34.01 | 88.61 | 24.89 | 26.63 | 25.76 | |
| Struct+2Way+Relation2019.11 | 18.02 | — | — | — | 35.61 | 33.45 | 58.84 | — | — | — | |
| EntailGen+QuesGen2019.11 | 17.76 | — | — | — | 35.98 | 33.63 | — | — | — | — | |
| DRGD2019.11 | 17.57 | — | — | — | 36.27 | 33.62 | — | — | — | — | |
| SEASS2019.11 | 17.54 | — | — | — | 36.15 | 33.63 | — | — | — | — | |
| PG Networks2019.11 | 16.92 | — | — | — | 34.19 | 31.81 | 58.32 | — | — | — | |
| SSTIAModel=SSTIA2023.06 | 16.9 | — | — | — | 36.55 | 33.25 | 88.72 | 23.52 | 24.01 | 23.76 | |
| Random Transformer2020.08 | 16.75 | — | — | — | 35.7 | 32.83 | — | — | — | — | |
| lvt5k-1sent2019.11 | 16.64 | — | — | — | 35.3 | 32.62 | — | — | — | — | |
| R3FModel=R3F2023.06 | 16.29 | — | — | — | 35.69 | 32.91 | 88.6 | 23.05 | 23.79 | 23.42 | |
| ProphetNetModel=ProphetNet2023.06 | 15.87 | — | — | — | 35.56 | 32.79 | 88.45 | 23.48 | 23.76 | 23.62 | |
| BARTModel=BART2023.06 | 15.64 | — | — | — | 35.23 | 32.52 | 87.33 | 22.42 | 22.32 | 22.37 | |
| Multi-Task w/ Entailment2019.11 | 15.35 | — | — | — | 32.75 | 30.82 | — | — | — | — | |
| Baseline2018.05 | — | 22 | 22 | 44 | — | — | — | — | — | — | |
| Soft Layer-Specific Multi-Task Summarizationmodel_type=3-way multi-task (MTL)2018.05 | — | 33 | 32 | 65 | — | — | — | — | — | — |