Abstractive Summarization on SAMSum
28.97ROUGE-2LEARNED SWA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| LEARNED SWA2024.11 | 28.97 | 52.93 | 44.04 | 41.98 | |
| BOMFHPBO Trajectory Type=Full2024.11 | 28.78 | 53.4 | 44.38 | 42.19 | |
| BOMFHPBO Trajectory Type=Freeze2024.11 | 28.61 | 53.07 | 44.4 | 42.03 | |
| HPBOLoRA Rank=642024.11 | 28.22 | 52.66 | 44.33 | 41.73 | |
| GREEDY SWA2024.11 | 28.06 | 53.4 | 43.31 | 41.49 | |
| BARTtraining=fine-tuned2021.06 | 27.82 | 52.27 | 47.92 | — | |
| SWA2024.11 | 27.61 | 51.81 | 43.55 | 40.99 | |
| Full Fine-tuning (All)Model Size=Mamba 1.4B, Params (%)=100.00, Type=None2025.03 | 27.3 | 51.2 | 42.9 | — | |
| Full Fine-tuning (S6)Model Size=Mamba 1.4B, Params (%)=4.46, Type=None2025.03 | 26.9 | 51.1 | 42.2 | — | |
| Prefix-TuningModel Size=Mamba 1.4B, Params (%)=12.81, Type=Prompt-based2025.03 | 26.5 | 50.6 | 42.1 | — | |
| State-offset Tuning (h)Model Size=Mamba 1.4B, Params (%)=0.23, Type=State-based2025.03 | 26.5 | 50.9 | 42.4 | — | |
| LoRAModel Size=Mamba 1.4B, Params (%)=0.46, Type=Parameter-based2025.03 | 26.4 | 50.5 | 42.2 | — | |
| State-offset Tuning (y)Model Size=Mamba 1.4B, Params (%)=0.01, Type=State-based2025.03 | 26.1 | 50.6 | 42 | — | |
| Initial State TuningModel Size=Mamba 1.4B, Params (%)=0.23, Type=State-based2025.03 | 26 | 50 | 41.3 | — | |
| BitFitModel Size=Mamba 1.4B, Params (%)=0.03, Type=Parameter-based2025.03 | 25.7 | 50.3 | 41.9 | — | |
| Gliwa et al.2021.06 | 25.6 | 49.3 | 47.7 | — | |
| Prompt TuningModel Size=Mamba 1.4B, Params (%)=0.01, Type=Prompt-based2025.03 | 25.6 | 50.1 | 41.6 | — | |
| CriSPOLLM=Claude 3 Sonnet, Few-shot Setting=3-shot2024.10 | 20.8 | 47.2 | 38.2 | — | |
| CriSPOLLM=Claude3 Sonnet, Strategy=Automatic Prompt Engineering, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 20.8 | — | — | — | |
| UNISUMMFew-shot samples=1002022.11 | 20.65 | 46.93 | 37.28 | — | |
| MultiBARTFew-shot samples=1002022.11 | 19.68 | 45.47 | 36.6 | — | |
| CriSPOLLM=Llama 3 8B, Few-shot Setting=0-shot, Optimizer=Claude 3 Sonnet2024.10 | 18.8 | 44.8 | 35.4 | — | |
| CriSPOLLM=Llama3 8B, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot, Optimizer=Claude3 Sonnet2024.10 | 18.8 | — | — | — | |
| CriSPOLLM=Claude Instant, Few-shot Setting=3-shot2024.10 | 18.7 | 45.7 | 36.2 | — | |
| CriSPOLLM=Claude Instant, Strategy=Automatic Prompt Engineering, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 18.7 | — | — | — | |
| UNISUMMFew-shot samples=102022.11 | 18.53 | 43.89 | 34.76 | — | |
| PEGASUSFew-shot samples=1002022.11 | 18.4 | 44.6 | 35.16 | — | |
| BART-PTFew-shot samples=1002022.11 | 17.82 | 43.39 | 34.42 | — | |
| GPT-3.5 (text-davinci-002)Few-shot samples=12022.11 | 17.65 | — | — | — | |
| Additional-scanModel Size=Mamba 1.4B, Params (%)=0.34, Type=Parameter-based2025.03 | 17.5 | 37.6 | 30.9 | — | |
| CriSPOLLM=Claude 3 Sonnet, Few-shot Setting=0-shot2024.10 | 17.1 | 43.4 | 34.3 | — | |
| CriSPOLLM=Claude3 Sonnet, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot2024.10 | 17.1 | — | — | — | |
| CriSPOLLM=Claude Instant, Few-shot Setting=0-shot2024.10 | 16.9 | 44.4 | 34.3 | — | |
| CriSPOLLM=Claude Instant, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot2024.10 | 16.9 | — | — | — | |
| PIAST (E)Time [m]=7372025.12 | 16.83 | 42.13 | 37.37 | — | |
| ManualLLM=Claude 3 Sonnet, Few-shot Setting=3-shot2024.10 | 16.6 | 41.1 | 31.3 | — | |
| ManualLLM=Claude3 Sonnet, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 16.6 | — | — | — | |
| MultiBARTFew-shot samples=102022.11 | 16.28 | 39.69 | 32.11 | — | |
| PRLTime [m]=28802025.12 | 16.17 | 42.47 | 37.73 | — | |
| PIASTTime [m]=34.482025.12 | 16.07 | 41.13 | 36.74 | — | |
| OPROLLM=Claude 3 Sonnet, Few-shot Setting=0-shot2024.10 | 14.7 | 39 | 30.1 | — | |
| OPROLLM=Llama 3 8B, Few-shot Setting=0-shot, Optimizer=Claude 3 Sonnet2024.10 | 14.7 | 39.3 | 30 | — | |
| OPROLLM=Claude3 Sonnet, Strategy=Automatic Prompt Engineering2024.10 | 14.7 | — | — | — | |
| OPROLLM=Llama3 8B, Strategy=Automatic Prompt Engineering, Optimizer=Claude3 Sonnet2024.10 | 14.7 | — | — | — | |
| GATime [m]=89.312025.12 | 14.47 | 39.69 | 35.84 | — | |
| ManualLLM=Claude Instant, Few-shot Setting=3-shot2024.10 | 14.3 | 37.8 | 28.8 | — | |
| ManualLLM=Claude Instant, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 14.3 | — | — | — | |
| CriSPOLLM=Mistral 7B, Few-shot Setting=3-shot2024.10 | 14.2 | 40 | 30.8 | — | |
| CriSPOLLM=Mistral 7B, Strategy=Automatic Prompt Engineering, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 14.2 | — | — | — | |
| BART-FTFew-shot samples=1002022.11 | 14.14 | 37.36 | 30.02 | — | |
| ManualLLM=Mistral 7B, Few-shot Setting=3-shot2024.10 | 14.1 | 39.5 | 30.3 | — | |
| ManualLLM=Mistral 7B, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 14.1 | — | — | — | |
| PEGASUSFew-shot samples=102022.11 | 13.79 | 38.58 | 30.37 | — | |
| OPROLLM=Mistral 7B, Few-shot Setting=0-shot2024.10 | 13.6 | 37.9 | 29 | — | |
| OPROLLM=Mistral 7B, Strategy=Automatic Prompt Engineering2024.10 | 13.6 | — | — | — | |
| OPROLLM=Claude Instant, Few-shot Setting=0-shot2024.10 | 13.4 | 38.1 | 28.7 | — | |
| OPROLLM=Claude Instant, Strategy=Automatic Prompt Engineering2024.10 | 13.4 | — | — | — | |
| APETime [m]=60.072025.12 | 12.97 | 37.12 | 33.32 | — | |
| BART-PTFew-shot samples=102022.11 | 12.96 | 35.53 | 28.26 | — | |
| ManualLLM=Claude 3 Sonnet, Few-shot Setting=0-shot2024.10 | 12.7 | 35.8 | 27 | — | |
| ManualLLM=Claude3 Sonnet, Evaluation Protocol=0-shot2024.10 | 12.7 | — | — | — | |
| DETime [m]=76.892025.12 | 12.53 | 33.91 | 31.05 | — | |
| CriSPOLLM=Mistral 7B, Few-shot Setting=0-shot2024.10 | 12.4 | 37.6 | 28.4 | — | |
| CriSPOLLM=Mistral 7B, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot2024.10 | 12.4 | — | — | — | |
| ManualLLM=Llama 3 8B, Few-shot Setting=0-shot2024.10 | 12.3 | 35.7 | 27.1 | — | |
| ManualLLM=Llama3 8B, Evaluation Protocol=0-shot2024.10 | 12.3 | — | — | — | |
| ManualLLM=Claude Instant, Few-shot Setting=0-shot2024.10 | 11.7 | 33.9 | 25.6 | — | |
| ManualLLM=Claude Instant, Evaluation Protocol=0-shot2024.10 | 11.7 | — | — | — | |
| MI2025.12 | 10.39 | 32.76 | 28.97 | — | |
| ManualLLM=Mistral 7B, Few-shot Setting=0-shot2024.10 | 10.2 | 32 | 24.1 | — | |
| ManualLLM=Mistral 7B, Evaluation Protocol=0-shot2024.10 | 10.2 | — | — | — | |
| BART-FTFew-shot samples=102022.11 | 4.23 | 18.07 | 14.7 | — | |
| PretrainedModel Size=Mamba 1.4B, Params (%)=0.00, Type=None2025.03 | 1.5 | 10.9 | 10.2 | — |