Abstractive Summarization on Gigaword (test)
39.81ROUGE-1OFA
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| OFA2022.02 | 39.81 | 20.66 | 37.11 | — | — | — | |
| OFA2023.02 | 39.81 | 20.66 | 37.11 | — | — | — | |
| UNIMO2022.02 | 39.71 | 20.37 | 36.88 | — | — | — | |
| UNIMO2023.02 | 39.71 | 20.37 | 36.88 | — | — | — | |
| mPLUG-22023.02 | 39.65 | 20.67 | 36.89 | — | — | — | |
| ProphetNet2020.01 | 39.55 | 20.27 | 36.57 | — | — | — | |
| ProphetNet2022.02 | 39.55 | 20.27 | 36.57 | — | — | — | |
| ProphetNet2023.02 | 39.55 | 20.27 | 36.57 | — | — | — | |
| ERNIE-GENLARGEData=430G, Params=340M2020.01 | 39.46 | 20.34 | 36.74 | — | — | — | |
| PALMModel size=LARGE2020.04 | 39.45 | 20.37 | 36.75 | — | — | — | |
| ERNIE-GENLARGEData=16G, Params=340M2020.01 | 39.25 | 20.25 | 36.53 | — | — | — | |
| ERNIE-GENModel size=LARGE2020.04 | 39.25 | 20.25 | 36.53 | — | — | — | |
| PEGASUS (HugeNews)Data=3.8T, Params=568M2020.01 | 39.12 | 19.86 | 36.24 | — | — | — | |
| PEGASUS2020.04 | 39.12 | 19.86 | 36.24 | — | — | — | |
| PEGASUS2022.02 | 39.12 | 19.86 | 36.24 | — | — | — | |
| PEGASUS2023.02 | 39.12 | 19.86 | 36.24 | — | — | — | |
| BISET2019.06 | 39.11 | 19.78 | 36.87 | — | — | — | |
| PEGASUS(C4)Data=750G, Params=568M2020.01 | 38.75 | 19.96 | 36.14 | — | — | — | |
| PALM2020.04 | 38.75 | 19.79 | 35.98 | — | — | — | |
| MASS2020.01 | 38.73 | 19.71 | 35.96 | — | — | — | |
| MASS2022.02 | 38.73 | 19.71 | 35.96 | — | — | — | |
| MASS2023.02 | 38.73 | 19.71 | 35.96 | — | — | — | |
| UniLMModel size=LARGE2020.04 | 38.45 | 19.45 | 35.75 | — | — | — | |
| UniLM2020.01 | 38.45 | 19.45 | 35.75 | — | — | — | |
| UniLM2022.02 | 38.45 | 19.45 | 35.75 | — | — | — | |
| UniLM2023.02 | 38.45 | 19.45 | 35.75 | — | — | — | |
| DMAN2021.03 | 38.28 | 19.46 | 35.46 | 31.06 | — | — | |
| MASS2020.04 | 38.13 | 19.81 | 35.62 | — | — | — | |
| BERTSHARE2022.02 | 38.13 | 19.81 | 35.62 | — | — | — | |
| BERTSHARE2023.02 | 38.13 | 19.81 | 35.62 | — | — | — | |
| Transformer2021.03 | 37.57 | 18.9 | 34.69 | 30.38 | — | — | |
| FTSum+2019.06 | 37.27 | 17.65 | 34.24 | — | — | — | |
| R³Sum+2019.06 | 37.04 | 19.03 | 34.46 | — | — | — | |
| Cao et al. (2018)Training=Supervised2019.07 | 37.04 | 19.03 | 34.46 | — | — | — | |
| Re3Sum2020.01 | 37.04 | 19.03 | 34.46 | — | — | — | |
| Contextual OracleType=Oracle performance, Selection=best summary among all finished hypothesis from beam search2019.07 | 37.03 | 15.46 | 33.23 | — | — | — | |
| OpenNMT2020.01 | 36.73 | 17.86 | 33.68 | — | — | — | |
| S2S+CGU+2019.06 | 36.3 | 18 | 33.8 | — | — | — | |
| SEASS2019.06 | 36.15 | 17.54 | 33.63 | — | — | — | |
| Open-NMT2019.06 | 34.07 | 16.35 | 31.78 | — | — | — | |
| RAS-Elman2019.06 | 33.78 | 15.97 | 31.15 | — | — | — | |
| seq2seqTraining=Supervised, Architecture=seq-to-seq model with attention2019.07 | 33.5 | 15.85 | 31.44 | — | — | — | |
| GPT-2medium_2.5#param.=2.12024.03 | 33.49 | 15.27 | 31.28 | 26.68 | — | — | |
| GPSTmedium#param.=2.12024.03 | 33.19 | 15.27 | 30.91 | 26.46 | — | — | |
| GPT-2medium#param.=2.02024.03 | 33.13 | 15.27 | 30.85 | 26.42 | — | — | |
| GPSTmedium w/o sync#param.=2.12024.03 | 32.83 | 15.06 | 30.59 | 26.16 | — | — | |
| GPT-2small_1.3#param.=1.12024.03 | 32.71 | 14.54 | 30.35 | 25.87 | — | — | |
| Featseq2seq2019.06 | 32.67 | 15.59 | 30.64 | — | — | — | |
| GPSTsmall#param.=1.052024.03 | 32.53 | 14.76 | 30.37 | 25.89 | — | — | |
| GPT-2small#param.=1.02024.03 | 32.45 | 14.84 | 30.37 | 25.88 | — | — | |
| GPSTsmall-w/o sync#param.=1.052024.03 | 32.34 | 14.69 | 29.98 | 25.67 | — | — | |
| ABS++2019.06 | 29.78 | 11.89 | 26.97 | — | — | — | |
| ABS+2019.06 | 29.55 | 11.32 | 26.42 | — | — | — | |
| Wang and Lee (2018)Training=Unsupervised2019.07 | 27.09 | 9.86 | 24.97 | — | — | — | |
| Contextual MatchTraining=Unsupervised2019.07 | 26.48 | 10.05 | 24.41 | — | — | — | |
| Lead-75CType=Prefix baseline2019.07 | 23.69 | 7.93 | 21.5 | — | — | — | |
| Schumann (2018)Training=Unsupervised2019.07 | 22.19 | 4.56 | 19.88 | — | — | — | |
| Lead-8Type=Prefix baseline2019.07 | 21.3 | 7.34 | 19.94 | — | — | — | |
| Baseline2021.08 | — | — | — | — | 76.12 | 83.33 | |
| DAE2021.08 | — | — | — | — | 78.23 | 86.33 | |
| Loss Truncation2021.08 | — | — | — | — | 79.55 | 87.17 | |
| Selector-FBbeta=0.52021.08 | — | — | — | — | 54.77 | 76.83 | |
| Selector-FBbeta=0.42021.08 | — | — | — | — | 59.79 | 81.67 | |
| Selector-FBbeta=0.32021.08 | — | — | — | — | 60.72 | 82 | |
| Selector-FBbeta=0.22021.08 | — | — | — | — | 68.38 | 86 | |
| Selector-FBbeta=0.12021.08 | — | — | — | — | 79.92 | 88 | |
| Selector-ROC2021.08 | — | — | — | — | 64.58 | 84.17 |