Abstractive Summarization on New York Times (test)
57.75ROUGE-1BRIO-Mul
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BRIO-MulBackbone=BART, Training Loss=Multi-task Loss, Training Mode=Fine-tuned2022.03 | 57.75 | 38.64 | 54.54 | |
| BRIO-CtrBackbone=BART, Training Loss=Contrastive Loss Only, Training Mode=Fine-tuned2022.03 | 55.98 | 36.54 | 52.51 | |
| BARTBackbone=BART, Evaluation Source=Current Paper Evaluation Script2022.03 | 55.78 | 36.61 | 52.6 | |
| (m7) DCA MLE+SEM+RLTraining loss=MLE+SEM+RL, Pointer mechanism=mpgen, Communication protocol=with-comm, Attention mechanism=contextual agent attention (caa), Agent configuration=3-agents2018.03 | 48.08 | 31.19 | 42.33 | |
| DCA2018.08 | 48.08 | 31.19 | 42.33 | |
| Bottom-Up Summarization2018.08 | 47.38 | 31.23 | 41.81 | |
| (m6) DCA MLE+SEMTraining loss=MLE+SEM, Pointer mechanism=mpgen, Communication protocol=with-comm, Attention mechanism=contextual agent attention (caa), Agent configuration=3-agents2018.03 | 47.3 | 30.5 | 41.06 | |
| RL, no intra-attentionTraining loss=RL, Attention mechanism=no intra-attention2018.03 | 47.22 | 30.51 | 43.27 | |
| ML+RL, no intra-attentionTraining loss=ML+RL, Attention mechanism=no intra-attention2018.03 | 47.03 | 30.72 | 43.1 | |
| ML+RLtraining=Maximum Likelihood + Reinforcement Learning2018.08 | 47.03 | 30.72 | 43.1 | |
| (m5) DCA MLE+SEMTraining loss=MLE+SEM, Pointer mechanism=mpgen, Communication protocol=with-comm, Agent configuration=3-agents2018.03 | 46.2 | 30.01 | 40.65 | |
| (m3) MLE+RLTraining loss=MLE+RL, Pointer mechanism=pgen, Communication protocol=no-comm, Agent configuration=1-agent2018.03 | 46.15 | 29.5 | 39.38 | |
| (m4) DCA MLE+SEMTraining loss=MLE+SEM, Pointer mechanism=pgen, Communication protocol=no-comm, Agent configuration=3-agents2018.03 | 45.84 | 28.23 | 39.32 | |
| Pointer-Generatorcoverage penalty=true2018.08 | 45.13 | 30.13 | 39.67 | |
| (m2) MLE+SEMTraining loss=MLE+SEM, Pointer mechanism=pgen, Communication protocol=no-comm, Agent configuration=1-agent2018.03 | 44.5 | 28.04 | 38.8 | |
| (m1) MLETraining loss=MLE, Pointer mechanism=pgen, Communication protocol=no-comm, Agent configuration=1-agent2018.03 | 44.28 | 26.01 | 37.87 | |
| ML, no intra-attentionTraining loss=ML, Attention mechanism=no intra-attention2018.03 | 44.26 | 27.43 | 40.41 | |
| MLtraining=Maximum Likelihood2018.08 | 44.26 | 27.43 | 40.41 |