Summarization on MLSUM (test)
52.3Score (De)Oracle
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| OracleModel Type=Extractive, Category=Upper Bound2022.04 | 52.3 | 35.78 | 37.69 | 29.8 | 45.78 | 40.27 | — | — | — | |
| MARGE (Train All)Learning Setting=Supervised, Model Type=Abstractive, Training Configuration=Train All2022.04 | 42.77 | 22.72 | 25.79 | 11.03 | 35.9 | 27.64 | — | — | — | |
| MARGE (Train One)Learning Setting=Supervised, Model Type=Abstractive, Training Configuration=Train One2022.04 | 42.6 | 22.31 | 25.91 | 10.85 | 36.09 | 27.55 | — | — | — | |
| mBERTSum-GenLearning Setting=Supervised, Model Type=Abstractive, Backbone=mBERT2022.04 | 42.01 | 20.44 | 25.09 | 9.48 | 32.94 | 25.99 | — | — | — | |
| XLMRSumLearning Setting=Supervised, Model Type=Extractive, Backbone=XLMR-Large2022.04 | 41.28 | 21.99 | 24.12 | 10.44 | 33.29 | 26.22 | — | — | — | |
| Pointer-GeneratorLearning Setting=Supervised, Model Type=Abstractive2022.04 | 35.08 | 17.67 | 23.58 | 5.71 | 32.59 | 22.99 | — | — | — | |
| NLSSumLearning Setting=Zero-Shot, Model Type=Extractive, Backbone=XLMR-Large2022.04 | 34.95 | 21.2 | 23.59 | 10.13 | 31.49 | 24.27 | — | — | — | |
| NLSSum-SepLearning Setting=Zero-Shot, Model Type=Extractive, Evaluation Strategy=Sep2022.04 | 34.21 | 21.24 | 23.92 | 10.09 | 31.68 | 24.23 | — | — | — | |
| Lead-2Model Type=Extractive, Category=Baseline, k=22022.04 | 33.09 | 13.7 | 19.69 | 5.94 | 28.9 | 20.26 | — | — | — | |
| XLMRSum-WRLearning Setting=Zero-Shot, Model Type=Extractive, Data Augmentation=Bilingual dictionary word replacement (WR)2022.04 | 32.37 | 21.03 | 23.67 | 9.34 | 30.1 | 23.3 | — | — | — | |
| XLMRSumLearning Setting=Zero-Shot, Model Type=Extractive, Backbone=XLMR-Large2022.04 | 32.05 | 19.49 | 22.2 | 8.7 | 27.64 | 22.02 | — | — | — | |
| MARGELearning Setting=Zero-Shot, Model Type=Abstractive2022.04 | 30.01 | 17.81 | 19.39 | 8.67 | 29.39 | 21.05 | — | — | — | |
| XLMRSum-MT w/ UbLearning Setting=Zero-Shot, Model Type=Extractive, Data Augmentation=Machine Translation (Ub)2022.04 | 29.7 | 21.18 | 23.62 | 9.37 | 24.27 | 21.63 | — | — | — | |
| XLMRSum-MT w/ UaLearning Setting=Zero-Shot, Model Type=Extractive, Data Augmentation=Machine Translation (Ua)2022.04 | 29.34 | 21.14 | 23.82 | 8.68 | 24.23 | 21.44 | — | — | — | |
| mBERTSumLearning Setting=Zero-Shot, Model Type=Extractive, Backbone=mBERT2022.04 | 17.36 | 17.27 | 19.64 | 8.37 | 19.3 | 16.39 | — | — | — | |
| mT5-LARGEModel Size=705M, #Training Tokens=1T2022.08 | — | — | — | — | — | — | 35.4 | 12.3 | 23.9 | |
| mT5-XLARGEModel Size=3B, #Training Tokens=1T2022.08 | — | — | — | — | — | — | 36.2 | 13.8 | 25 | |
| PaLM540BModel Size=540B, #Training Tokens=500B2022.08 | — | — | — | — | — | — | 33.1 | 12 | 22.6 | |
| Z-Code++LARGEModel Size=710M, #Training Tokens=500B2022.08 | — | — | — | — | — | — | 36.8 | 14.8 | 25.8 |