Summarization on Gigaword
36.88ROUGE-LUNIMO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| UNIMOSize=large2020.12 | 36.88 | 39.71 | 20.37 | |
| RoBERTaSize=large, Implementation=UNIMO architecture with RoBERTa weights2020.12 | 36.58 | 39.32 | 20.01 | |
| UNIMOSize=base2020.12 | 36.27 | 38.8 | 19.99 | |
| RoBERTaSize=base, Implementation=UNIMO architecture with RoBERTa weights2020.12 | 36.04 | 38.65 | 19.66 | |
| UniLMSize=large2020.12 | 35.75 | 38.45 | 19.45 | |
| UNIMO (w/o single-modal)Size=base, Note=Removing single-modal learning process2020.12 | 35.41 | 38.06 | 18.91 | |
| Soft Layer-Specific Multi-Task SummarizationAuxiliary Task=Entailment + Question Generation2018.05 | 33.63 | 35.98 | 17.76 | |
| Soft Layer-Specific Multi-Task SummarizationAuxiliary Task=Entailment Generation2018.05 | 33.19 | 35.45 | 17.16 | |
| GENICLBackbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 33 | — | — | |
| Soft Layer-Specific Multi-Task SummarizationAuxiliary Task=Question Generation2018.05 | 32.97 | 35.48 | 17.31 | |
| BM25Backbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 32.5 | — | — | |
| LLM-RBackbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 32.5 | — | — | |
| Centroid Transformer (Mean-Pooling Initialization)MACS(M)=262.9, MBS/GPU=230, initialization=mean-pooling2021.02 | 32.415 | 34.651 | 16.468 | |
| EPRBackbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 32.4 | — | — | |
| 2-Layer Pointer Baseline2018.05 | 32.03 | 34.26 | 16.4 | |
| E5baseBackbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 31.9 | — | — | |
| SBERTBackbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 31.7 | — | — | |
| RAS-EI2018.05 | 31.15 | 33.78 | 15.97 | |
| Pasunuru et al. (2017)2018.05 | 30.82 | 32.75 | 15.35 | |
| TransformerMACS(M)=523.2, MBS/GPU=1922021.02 | 30.771 | 32.987 | 15.286 | |
| lvt2k2018.05 | 30.64 | 32.67 | 15.59 | |
| CBDSBackbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 29.2 | — | — | |
| HTLMzero-shot=true, prompting strategy=manually engineered, size hints=true2021.07 | 28.6 | 31.61 | 10.8 | |
| Centroid Transformer (Random Initialization)MACS(M)=262.9, MBS/GPU=230, initialization=random sampling2021.02 | 27.823 | 30.31 | 12.752 | |
| RandomBackbone=LLaMA-7B, Evaluation Setting=Few-shot (ICL)2025.05 | 27.2 | — | — | |
| ABS+2018.05 | 26.96 | 29.76 | 11.88 | |
| HTLMzero-shot=true, prompting strategy=autoprompting, size hints=true2021.07 | 26.49 | 28.73 | 11.31 | |
| UPRISE2024.02 | 25.8 | — | — | |
| Se²2024.02 | 25.8 | — | — | |
| HTLMzero-shot=true, prompting strategy=autoprompting, size hints=false2021.07 | 24.57 | 27.56 | 10.17 | |
| BM252024.02 | 23.3 | — | — | |
| Best-of-102024.02 | 21.8 | — | — | |
| PEGASUSzero-shot=true2021.07 | 20.2 | 23.39 | 7.59 | |
| Instructor2024.02 | 19.9 | — | — | |
| SBERT2024.02 | 19.4 | — | — | |
| Random2024.02 | 18.1 | — | — | |
| Zero-shotBackbone=LLaMA-7B, Evaluation Setting=Zero-shot2025.05 | 15.4 | — | — | |
| Zero-shot2024.02 | 1.4 | — | — |