Summarization on XSum (ROUGE-1, ROUGE-2, ROUGE-L)
27.1ROUGE-2SOTA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SOTAEvaluation Protocol=Finetuning2022.08 | 27.1 | — | — | |
| AlexaTM 20BEvaluation Protocol=Finetuning, Model Scale=20B2022.08 | 24.16 | — | — | |
| PaLMEvaluation Protocol=Finetuning, Model Scale=540B2022.08 | 21.2 | — | — | |
| T5 XXLEvaluation Protocol=Finetuning, Model Scale=XXL2022.08 | 21 | — | — | |
| PaLMEvaluation Protocol=Finetuning, Model Scale=62B2022.08 | 18.5 | — | — | |
| PaLMEvaluation Protocol=Finetuning, Model Scale=8B2022.08 | 16.3 | — | — | |
| PaLMEvaluation Protocol=1-shot, Model Scale=540B2022.08 | 12.2 | — | — | |
| AlexaTM 20BEvaluation Protocol=1-shot, Model Scale=20B2022.08 | 11.69 | — | — | |
| PaLMEvaluation Protocol=1-shot, Model Scale=62B2022.08 | 11.2 | — | — | |
| PaLMEvaluation Protocol=1-shot, Model Scale=8B2022.08 | 7.9 | — | — | |
| HTLMzero-shot=true, prompting strategy=manually engineered, size hints=true2021.07 | 4.12 | 22.34 | 14.56 | |
| HTLMzero-shot=true, prompting strategy=autoprompting, size hints=true2021.07 | 3.41 | 17.14 | 13.43 | |
| PEGASUSzero-shot=true2021.07 | 3 | 19.27 | 12.72 | |
| HTLMzero-shot=true, prompting strategy=autoprompting, size hints=false2021.07 | 2.54 | 15.15 | 10.91 |