Summarization on SamSum (test)
53.4ROUGE-1GREEDY SWA
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GREEDY SWABackbone=LLaMA2-7B, Rank=642024.11 | 53.4 | 28.06 | 43.31 | 41.49 | — | |
| BOMFBackbone=LLaMA2-7B, Rank=642024.11 | 53.4 | 28.78 | 44.38 | 42.19 | — | |
| BOMFBackbone=LLaMA2-7B, Rank=42024.11 | 53.07 | 28.61 | 44.4 | 42.03 | — | |
| GREEDY SWABackbone=LLaMA2-7B, Rank=42024.11 | 53.06 | 28.99 | 44.03 | 42.03 | — | |
| LEARNED SWABackbone=LLaMA2-7B, Rank=642024.11 | 52.93 | 28.97 | 44.04 | 41.98 | — | |
| HPBOBackbone=LLaMA2-7B, Rank=642024.11 | 52.66 | 28.22 | 44.33 | 41.73 | — | |
| HPBOBackbone=LLaMA2-7B, Rank=42024.11 | 52.25 | 27.52 | 44.03 | 41.27 | — | |
| SWABackbone=LLaMA2-7B, Rank=42024.11 | 52.21 | 27.58 | 44.04 | 41.28 | — | |
| LEARNED SWABackbone=LLaMA2-7B, Rank=42024.11 | 52.1 | 26.76 | 44.13 | 41 | — | |
| SWABackbone=LLaMA2-7B, Rank=642024.11 | 51.81 | 27.61 | 43.55 | 40.99 | — | |
| Mistral 7BFramework=Adaptive HPF, Threshold=0.22024.06 | 28.335 | — | — | — | — | |
| Mistral 7BFramework=Adaptive HPF, Threshold=0.152024.06 | 26.91 | — | — | — | — | |
| Llama-3 8BFramework=Adaptive HPF, Threshold=0.22024.06 | 24.981 | — | — | — | — | |
| Llama-3 8BFramework=Adaptive HPF, Threshold=0.152024.06 | 24.407 | — | — | — | — | |
| Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.22024.06 | 21.58 | — | — | — | — | |
| Phi-3 3.8BFramework=Adaptive HPF, Threshold=0.152024.06 | 20.586 | — | — | — | — | |
| Gemma 7BFramework=Adaptive HPF, Threshold=0.22024.06 | 16.179 | — | — | — | — | |
| Gemma 7BFramework=Adaptive HPF, Threshold=0.152024.06 | 16.023 | — | — | — | — | |
| CriSPOLLM=Claude Instant, Shots=02024.10 | — | — | — | — | 34.3 | |
| CriSPOLLM=Claude Instant, Shots=32024.10 | — | — | — | — | 36.2 | |
| CriSPOLLM=Claude3 Sonnet, Shots=02024.10 | — | — | — | — | 34.3 | |
| CriSPOLLM=Claude3 Sonnet, Shots=32024.10 | — | — | — | — | 38.2 | |
| CriSPOLLM=Mistral 7B, Shots=02024.10 | — | — | — | — | 28.4 | |
| CriSPOLLM=Mistral 7B, Shots=32024.10 | — | — | — | — | 30.8 | |
| CriSPOLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.10 | — | — | — | — | 35.4 | |
| GPT-3 + LoRANumber of Parameters=175B2022.08 | — | 29.8 | — | — | — | |
| ManualLLM=Claude Instant, Shots=02024.10 | — | — | — | — | 25.6 | |
| ManualLLM=Claude Instant, Shots=32024.10 | — | — | — | — | 28.8 | |
| ManualLLM=Claude3 Sonnet, Shots=02024.10 | — | — | — | — | 27 | |
| ManualLLM=Claude3 Sonnet, Shots=32024.10 | — | — | — | — | 31.3 | |
| ManualLLM=Mistral 7B, Shots=02024.10 | — | — | — | — | 24.1 | |
| ManualLLM=Mistral 7B, Shots=32024.10 | — | — | — | — | 30.3 | |
| ManualLLM=Llama3 8B, Shots=02024.10 | — | — | — | — | 27.1 | |
| Narayan et al.status=previous state-of-the-art2022.05 | — | 28.3 | — | — | — | |
| OPROLLM=Claude Instant, Shots=02024.10 | — | — | — | — | 28.7 | |
| OPROLLM=Claude3 Sonnet, Shots=02024.10 | — | — | — | — | 30.1 | |
| OPROLLM=Mistral 7B, Shots=02024.10 | — | — | — | — | 29 | |
| OPROLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.10 | — | — | — | — | 30 | |
| PEGASUS_LARGENumber of Parameters=470M2022.08 | — | 26.3 | — | — | — | |
| UL20Bmode=supervised finetuning2022.05 | — | 29.6 | — | — | — | |
| Z-Code++_LARGENumber of Parameters=710M2022.08 | — | 30.3 | — | — | — |