Summarization on Newsroom (test)
74ROUGE-2TLM+E (G,G)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| TLM+E (G,G)Type=Oracle2019.09 | 74 | 78.8 | 77.8 | — | |
| Gold ExtType=Oracle2019.09 | 64.5 | 68.1 | 67.3 | — | |
| ExconsummType=Mix2019.09 | 62.9 | 68.4 | 67.3 | — | |
| TLM+E (G,M)Type=Mix2019.09 | 57.3 | 63.3 | 61.8 | — | |
| Sent-PTRType=Ext2019.09 | 55.2 | 60.7 | 59.7 | — | |
| Lead-3Type=Ext2019.09 | 49 | 53 | 52.4 | — | |
| Sent-CLFType=Ext2019.09 | 47 | 53 | 52.1 | — | |
| TLMType=Abs2019.09 | 39.7 | 49.8 | 47.4 | — | |
| PEGASUS2022.10 | 33.51 | 45.15 | 41.33 | — | |
| Prior SOTA2022.08 | 33.5 | — | — | — | |
| PEGASUS_LARGENumber of Parameters=470M2022.08 | 33.5 | — | — | — | |
| SEASON2022.10 | 33.37 | 46 | 42.03 | — | |
| Z-Code++_LARGENumber of Parameters=710M2022.08 | 33.1 | — | — | — | |
| BARTImplementation=Reproduced by authors2022.10 | 33.05 | 45.5 | 41.69 | — | |
| Pointer-genType=Mix2019.09 | 27.9 | 39.1 | 36.2 | — | |
| CopyTransTraining Data=Newsroom, Variant=best-abs2019.11 | 24.14 | 45.93 | 42.51 | 66.2 | |
| CopyTransTraining Data=Newsroom, Variant=pure-ext2019.11 | 21.81 | 43.21 | 40.05 | 63.68 | |
| LEAD-32022.10 | 21.27 | 30.49 | 28.42 | — | |
| Lead-3 baselineLearning Paradigm=Supervised, Extractive=true2021.05 | 21.1 | 32 | 29.6 | — | |
| Struct+2Way+RelationTraining Data=Newsroom2019.11 | 20.44 | 40.54 | 37.4 | 62.05 | |
| TextRankType=Ext2019.09 | 19.7 | 32.4 | 28.7 | — | |
| PG NetworksTraining Data=Newsroom2019.11 | 19.51 | 39.86 | 36.61 | 62.01 | |
| CopyTransTraining Data=Gigaword, Variant=best-abs2019.11 | 19.11 | 40.89 | 37.6 | 62.74 | |
| CopyTransTraining Data=Gigaword, Variant=pure-ext2019.11 | 17.32 | 39.44 | 36.1 | 61 | |
| PG + CoverageLearning Paradigm=Supervised, Extractive=false2021.05 | 13.3 | 27.5 | 23.5 | — | |
| Point-Generator2022.10 | 13.25 | 26.02 | 22.43 | — | |
| TextRankLearning Paradigm=Unsupervised, Extractive=true2021.05 | 10.1 | 24.5 | 20.1 | — | |
| TLM+E (G,G)Type=Oracle2019.09 | 9.6 | 24.5 | 20.8 | — | |
| Summary Loop 24Learning Paradigm=Unsupervised, Extractive=false2021.05 | 9.6 | 27 | 26.4 | — | |
| TLMType=Abs2019.09 | 6.9 | 20.4 | 17.1 | — | |
| TLM+E (G,M)Type=Mix2019.09 | 6.5 | 20.1 | 16.6 | — | |
| Gold ExtType=Oracle2019.09 | 5.2 | 21.9 | 16.3 | — | |
| ExconsummType=Mix2019.09 | 3.1 | 17.1 | 14.1 | — | |
| Sent-PTRType=Ext2019.09 | 2.8 | 15.9 | 13 | — | |
| Sent-CLFType=Ext2019.09 | 2.7 | 15.4 | 12.8 | — | |
| Lead-3Type=Ext2019.09 | 2.4 | 13.7 | 11.2 | — | |
| Pointer-genType=Mix2019.09 | 2.3 | 14.7 | 11.4 | — | |
| TextRankType=Ext2019.09 | 1.9 | 13.5 | 10.5 | — | |
| Seq2SeqType=Abs2019.09 | 1.1 | 6.2 | 5.7 | — | |
| Seq2SeqType=Abs2019.09 | 0.3 | 6.1 | 5.4 | — |