Extractive Summarization on CNN/Daily Mail (test)
58ROUGE-1Human
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Human2019.05 | 58 | 15 | 20 | 0 | 3 | 3 | 1.85 | |
| ORACLEselection_rule=greedy selection to maximize ROUGE2019.03 | 52.59 | 31.24 | 48.87 | — | — | — | — | |
| BERTSUM+Transformerbackbone=BERT, decoder=Transformer2019.03 | 43.25 | 20.24 | 39.63 | — | — | — | — | |
| BERTSUM+Classifierbackbone=BERT, decoder=linear classifier2019.03 | 43.23 | 20.22 | 39.6 | — | — | — | — | |
| BERTSUM+LSTMbackbone=BERT, decoder=LSTM2019.03 | 43.22 | 20.17 | 39.59 | — | — | — | — | |
| ITS2018.09 | 42.4 | 17.4 | 34.1 | — | — | — | — | |
| Cheng et.al2018.09 | 42.2 | 17.3 | 34.8 | — | — | — | — | |
| SummaRuNNer2018.09 | 42 | 16.9 | 34.1 | — | — | — | — | |
| DCA*type=abstractive summarization system, agents=multiple2019.03 | 41.69 | 19.47 | 37.92 | — | — | — | — | |
| NEUSUM*type=extractive summarization system, strategy=joint scoring and sentence selection2019.03 | 41.59 | 19.01 | 37.98 | — | — | — | — | |
| BANDITSUM2018.09 | 41.5 | 18.7 | 37.6 | — | — | — | — | |
| Hybrid MemNet2018.09 | 41.4 | 16.7 | 33.2 | — | — | — | — | |
| RNEScoherence=false2018.09 | 41.3 | 18.9 | 37.6 | — | — | — | — | |
| RNEScoherence=without2018.04 | 41.25 | 18.87 | 37.75 | — | — | — | — | |
| REFRESH*type=extractive summarization system, optimization=globally optimizing ROUGE via RL2019.03 | 41 | 18.8 | 37.7 | — | — | — | — | |
| RNEScoherence=with2018.04 | 40.95 | 18.63 | 37.41 | — | — | — | — | |
| Transformerlayers=6, hidden size=512, feed-forward size=2048, pretrained=false, initialization=randomly initialized2019.03 | 40.9 | 18.02 | 37.17 | — | — | — | — | |
| Lead-32018.09 | 40.5 | 14.9 | 32.6 | — | — | — | — | |
| LEADtype=extractive baseline, selection_rule=first 3 sentences2019.03 | 40.42 | 17.62 | 36.67 | — | — | — | — | |
| REFRESH2018.09 | 40.3 | 15.1 | 32.9 | — | — | — | — | |
| Lead-3implementation=ours2018.09 | 40 | 17.5 | 36.2 | — | — | — | — | |
| Refresh2018.09 | 40 | 18.2 | 36.6 | — | — | — | — | |
| Nallapati et al. 20172018.04 | 39.6 | 16.2 | 35.3 | — | — | — | — | |
| Leadsource=Narayan et al., 20182018.09 | 39.6 | 17.7 | 36.2 | — | — | — | — | |
| SummaRuNNer2018.09 | 39.6 | 16.2 | 35.3 | — | — | — | — | |
| See et al. 20172018.04 | 39.53 | 17.28 | 35.38 | — | — | — | — | |
| PGN*type=abstractive summarization system, architecture=encoder-decoder2019.03 | 39.53 | 17.28 | 37.98 | — | — | — | — | |
| DQN2018.09 | 39.4 | 16.1 | 35.6 | — | — | — | — | |
| Lead-32018.04 | 39.2 | 15.7 | 35.5 | — | — | — | — | |
| NES2018.04 | 37.75 | 17.04 | 33.92 | — | — | — | — | |
| Nallapati et al. 20162018.04 | 35.4 | 13.3 | 32.6 | — | — | — | — | |
| HIBERT_M2019.05 | 30 | 35 | 25 | 10 | 0 | 0 | 2.15 | |
| BERTnumber of parameters=110M2019.05 | 13 | 37 | 32 | 15 | 3 | 0 | 2.58 | |
| DCA2019.05 | 8 | 15 | 18 | 20 | 15 | 23 | 3.88 | |
| Latent2019.05 | 5 | 33 | 28 | 20 | 13 | 0 | 3.03 | |
| Lead32019.05 | 3 | 18 | 15 | 30 | 30 | 3 | 3.75 |