Text Summarization on DUC 2004 (test)
38.57ROUGE-1RegSum
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| RegSum2017.06 | 38.57 | 9.75 | — | — | — | — | — | — | |
| GRU+GCN: Personalized Discourse Graphgraph=Personalized Discourse Graph (PDG)2017.06 | 38.23 | 9.48 | — | — | — | — | — | — | |
| GreedyKL2017.06 | 37.98 | 8.53 | — | — | — | — | — | — | |
| CLASSY042017.06 | 37.62 | 8.96 | — | — | — | — | — | — | |
| GRU+GCN: ADG from G-Flowgraph=Approximate Discourse Graph (ADG)2017.06 | 37.41 | 8.97 | — | — | — | — | — | — | |
| GRU+GCN: Cosine Similarity Graphgraph=Cosine Similarity Graph2017.06 | 37.33 | 8.78 | — | — | — | — | — | — | |
| CLASSY112017.06 | 37.22 | 9.2 | — | — | — | — | — | — | |
| GRUgraph=none2017.06 | 36.64 | 8.47 | — | — | — | — | — | — | |
| Centroid2017.06 | 36.41 | 7.97 | — | — | — | — | — | — | |
| SVR2017.06 | 36.18 | 9.34 | — | — | — | — | — | — | |
| Cont. LexRank2017.06 | 35.95 | 7.47 | — | — | — | — | — | — | |
| TsSum2017.06 | 35.88 | 8.15 | — | — | — | — | — | — | |
| G-Flow2017.06 | 35.3 | 8.27 | — | — | — | — | — | — | |
| FreqSum2017.06 | 35.3 | 8.11 | — | — | — | — | — | — | |
| PRIMERAzero-shot=true, output length limit=1282021.10 | 35.1 | 7.2 | 17.9 | — | — | — | — | — | |
| WDROPPosition=both2021.04 | 33.06 | 11.45 | 28.51 | — | — | — | — | — | |
| Transformer+LRPE+PE+Re-ranking (Ensemble)Positional Encoding=LRPE, additional PE=true, Re-ranking=true, Ensemble size=5 models2019.04 | 32.85 | 11.78 | 28.52 | — | — | — | — | — | |
| REP(SS)Position=dec2021.04 | 32.83 | 11.41 | 28.14 | — | — | — | — | — | |
| w/o perturbation2021.04 | 32.8 | 11.55 | 28.26 | — | — | — | — | — | |
| REP(SIM)Position=both2021.04 | 32.8 | 11.55 | 28.28 | — | — | — | — | — | |
| REP(SIM)+WDROPPosition=both2021.04 | 32.8 | 11.73 | 28.46 | — | — | — | — | — | |
| PEGASUSzero-shot=true, source=our run, output length limit=1282021.10 | 32.7 | 7.4 | 17.6 | — | — | — | — | — | |
| Transformer+LRPE+PE+ALONED_inter=1024, Embed=1.0M, Encoding Type=Real number2020.04 | 32.57 | 11.63 | 28.24 | — | — | — | — | — | |
| Takase and Kobayashi2021.04 | 32.57 | 11.63 | 28.24 | — | — | — | — | — | |
| REP(UNI)Position=both2021.04 | 32.56 | 11.48 | 28.21 | — | — | — | — | — | |
| Transformer+LRPE+PE+ALONED_inter=1024, Embed=1.0M, Encoding Type=Binary2020.04 | 32.51 | 11.48 | 28.08 | — | — | — | — | — | |
| Transformer+LRPE+PE+Re-rankingPositional Encoding=LRPE, additional PE=true, Re-ranking=true2019.04 | 32.29 | 11.49 | 28.03 | — | — | — | — | — | |
| Transformer+LRPE+PEEmbed=8.3M2020.04 | 32.29 | 11.49 | 28.03 | — | — | — | — | — | |
| Takase and Okazaki2021.04 | 32.29 | 11.49 | 28.03 | — | — | — | — | — | |
| Suzuki and Nagata2019.04 | 32.28 | 10.54 | 27.8 | — | — | — | — | — | |
| LSTM EncDec+WFEEmbed=37.7M2020.04 | 32.28 | 10.54 | 27.8 | — | — | — | — | — | |
| Suzuki and Nagata2021.04 | 32.28 | 10.54 | 27.8 | — | — | — | — | — | |
| REP(UNI)+WDROPPosition=both2021.04 | 32.15 | 11.58 | 28.01 | — | — | — | — | — | |
| Transformer+LRPE+PE+ALONED_inter=512, Embed=0.5M, Encoding Type=Real number2020.04 | 31.96 | 11.5 | 27.74 | — | — | — | — | — | |
| DRGD2017.08 | 31.79 | 10.75 | 27.48 | — | — | — | — | — | |
| Li et al. (2017)2019.04 | 31.79 | 10.75 | 27.48 | — | — | — | — | — | |
| Transformer+LDPE+PE+Re-rankingPositional Encoding=LDPE, additional PE=true, Re-ranking=true2019.04 | 31.65 | 11.25 | 27.46 | — | — | — | — | — | |
| Transformer+LRPE+PE+ALONED_inter=512, Embed=0.5M, Encoding Type=Binary2020.04 | 31.6 | 11.12 | 27.25 | — | — | — | — | — | |
| Transformer+LRPE+PEPositional Encoding=LRPE, additional PE=true2019.04 | 31.1 | 11.05 | 27.25 | — | — | — | — | — | |
| Transformer+LDPE+PEPositional Encoding=LDPE, additional PE=true2019.04 | 31 | 10.78 | 27.02 | — | — | — | — | — | |
| Transformer+LDPEPositional Encoding=LDPE2019.04 | 30.95 | 10.53 | 26.79 | — | — | — | — | — | |
| Transformer+LRPEPositional Encoding=LRPE2019.04 | 30.74 | 10.83 | 26.69 | — | — | — | — | — | |
| Concept pointeroptimization=DS2019.10 | 30.39 | 10.78 | 27.53 | — | — | — | — | — | |
| Fan et al.Task control=output length2019.04 | 30 | 10.27 | 26.43 | — | — | — | — | — | |
| LenInittype=Baseline2019.04 | 29.78 | 11.05 | 26.49 | — | — | — | — | — | |
| Concept pointeroptimization=RL2019.10 | 29.34 | 9.84 | 26.6 | — | — | — | — | — | |
| Li et al. (2018)2019.04 | 29.33 | 10.24 | 25.24 | — | — | — | — | — | |
| Seq2seq-Sel.-MTL-ERAM2019.10 | 29.33 | 10.24 | 25.24 | — | — | — | — | — | |
| REFERENCE2015.09 | 29.21 | 8.38 | 24.46 | — | — | — | — | — | |
| SEASSDecoding Strategy=beam2017.04 | 29.21 | 9.56 | 25.51 | — | — | — | — | — | |
| Zhou et al.2019.04 | 29.21 | 9.56 | 25.51 | — | — | — | — | — | |
| SEASS2019.10 | 29.21 | 9.56 | 25.51 | — | — | — | — | — | |
| Concept pointeroptimization=baseline2019.10 | 29.17 | 10.38 | 26.34 | — | — | — | — | — | |
| CAs2sDecoding Strategy=greedy2017.04 | 29.13 | 7.62 | 23.92 | — | — | — | — | — | |
| RAS-ElmanTraining dataset=Gigaword, Evaluation protocol=limited-length Rouge recall at 75 bytes2016.02 | 28.97 | 8.26 | 24.06 | — | — | — | — | — | |
| CAs2sDecoding Strategy=beam2017.04 | 28.97 | 8.26 | 24.06 | — | — | — | — | — | |
| RAS-Elman2017.08 | 28.97 | 8.26 | 24.06 | — | — | — | — | — | |
| RAS-Elman2019.10 | 28.97 | 8.26 | 24.06 | — | — | — | — | — | |
| SEASSDecoding Strategy=greedy2017.04 | 28.68 | 8.55 | 25.04 | — | — | — | — | — | |
| LCtype=Baseline2019.04 | 28.68 | 10.79 | 25.72 | — | — | — | — | — | |
| words-lvt5k-1sentTraining dataset=Gigaword, Tuned on DUC validation=false, Source input=1 sentence, Evaluation protocol=limited-length Rouge recall at 75 bytes2016.02 | 28.61 | 9.42 | 25.24 | — | — | — | — | — | |
| lvt5k-1senvocab size=5k, utilize linguistic features=true2017.08 | 28.61 | 9.42 | 25.24 | — | — | — | — | — | |
| lvt5k-lsent2019.10 | 28.61 | 9.42 | 25.24 | — | — | — | — | — | |
| Seq2seq+attimplementation=re-implementation2019.10 | 28.57 | 9.31 | 24.81 | — | — | — | — | — | |
| Luong-NMTDecoding Strategy=beam2017.04 | 28.55 | 8.79 | 24.43 | — | — | — | — | — | |
| Luong-NMT2019.10 | 28.55 | 8.79 | 24.43 | — | — | — | — | — | |
| NACCSetting=Unsupervised, Approach Category=NAR, Strategy=length control2022.05 | 28.37 | 7.74 | 24.3 | — | — | — | 6.68 | 0.03 | |
| words-lvt2k-1sentTraining dataset=Gigaword, Tuned on DUC validation=false, Source input=1 sentence, Evaluation protocol=limited-length Rouge recall at 75 bytes2016.02 | 28.35 | 9.46 | 24.59 | — | — | — | — | — | |
| Feats2sDecoding Strategy=beam2017.04 | 28.35 | 9.46 | 24.59 | — | — | — | — | — | |
| lvt2k-1senvocab size=2k, utilize linguistic features=true2017.08 | 28.35 | 9.46 | 24.59 | — | — | — | — | — | |
| Pointer-generatorimplementation=re-implementation2019.10 | 28.28 | 10.04 | 25.69 | — | — | — | — | — | |
| ABS+type=Attention-Based Summarization, tuning=MERT tuning step (Z-MERT)2015.09 | 28.18 | 8.49 | 23.81 | — | — | — | — | — | |
| ABS+Training dataset=Gigaword + DUC 2003 tuning, Tuned on DUC validation=true, Evaluation protocol=limited-length Rouge recall at 75 bytes2016.02 | 28.18 | 8.49 | 23.81 | — | — | — | — | — | |
| ABS+Decoding Strategy=beam2017.04 | 28.18 | 8.49 | 23.81 | — | — | — | — | — | |
| ABS+2017.08 | 28.18 | 8.49 | 23.81 | — | — | — | — | — | |
| Rush et al.2019.04 | 28.18 | 8.49 | 23.81 | — | — | — | — | — | |
| ABS+2019.10 | 28.18 | 8.46 | 23.81 | — | — | — | — | — | |
| ABSEmbed=42.1M2020.04 | 28.18 | 8.49 | 23.81 | — | — | — | — | — | |
| Rush et al.2021.04 | 28.18 | 8.49 | 23.81 | — | — | — | — | — | |
| s2s+attDecoding Strategy=beam2017.04 | 28.13 | 9.25 | 24.76 | — | — | — | — | — | |
| RAS-LSTM2017.08 | 27.41 | 7.69 | 23.06 | — | — | — | — | — | |
| s2s+attDecoding Strategy=greedy2017.04 | 27.03 | 7.89 | 23.8 | — | — | — | — | — | |
| LenEmb2017.08 | 26.73 | 8.39 | 23.88 | — | — | — | — | — | |
| Kikuchi et al.Task control=output length2019.04 | 26.73 | 8.39 | 23.88 | — | — | — | — | — | |
| NAUSvariant=length control, Time=0.048, Speedup=257x2022.05 | 26.71 | 7.68 | 23.06 | — | — | 0.4 | — | — | |
| ABStype=Attention-Based Summarization2015.09 | 26.55 | 7.06 | 22.05 | — | — | — | — | — | |
| ABSTraining dataset=Gigaword, Evaluation protocol=limited-length Rouge recall at 75 bytes2016.02 | 26.55 | 7.06 | 22.05 | — | — | — | — | — | |
| ABSDecoding Strategy=beam2017.04 | 26.55 | 7.06 | 22.05 | — | — | — | — | — | |
| ABS2017.08 | 26.55 | 7.06 | 22.05 | — | — | — | — | — | |
| NAUSvariant=truncate, Time=0.005, Speedup=2463x2022.05 | 26.52 | 7.88 | 22.91 | — | — | 0.26 | — | — | |
| MOSES+2015.09 | 26.5 | 8.13 | 22.85 | — | — | — | — | — | |
| MOSES+2017.08 | 26.5 | 8.13 | 22.85 | — | — | — | — | — | |
| NACCSetting=Unsupervised, Approach Category=NAR, Strategy=truncate2022.05 | 26.43 | 7.86 | 22.66 | — | — | — | 3.22 | 0.012 | |
| Char-constrained searchSetting=Unsupervised, Approach Category=Search2022.05 | 26.3 | 7.95 | 22.78 | — | — | — | 3.3 | 31.54 | |
| Su et al. (2021)Time=0.022, Speedup=559x2022.05 | 26.25 | 7.66 | 22.83 | — | — | -0.31 | — | — | |
| Transformertype=Baseline2019.04 | 26.15 | 9.14 | 23.19 | — | — | — | — | — | |
| Schumann et al. (2020)Setup=Our replication, Time=12.314, Speedup=1x2022.05 | 26.14 | 8.03 | 22.88 | — | — | 0 | — | — | |
| Schumann et al.Setting=Unsupervised, Approach Category=Search, Strategy=truncate2022.05 | 26.09 | 8.03 | 22.86 | — | — | — | 3.25 | 30.362 | |
| Schumann et al. (2020)2022.05 | 26.04 | 8.06 | 22.9 | — | — | -0.05 | — | — | |
| TOPIARY2015.09 | 25.12 | 6.46 | 20.12 | — | — | — | — | — |