Summarization on CNN (ROUGE R1, R2, RL)
42.2ROUGE-1CriSPO
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CriSPOLLM=Claude 3 Sonnet, Few-shot Setting=0-shot2024.10 | 42.2 | 17.3 | 27.9 | |
| CriSPOLLM=Claude Instant, Few-shot Setting=3-shot (ICL with example selection)2024.10 | 42.1 | 17 | 27.4 | |
| CriSPOLLM=Claude 3 Sonnet, Few-shot Setting=3-shot (ICL with example selection)2024.10 | 41.6 | 16.3 | 27.1 | |
| CriSPOLLM=Llama 3 8B, Few-shot Setting=0-shot, Optimizer=Claude 3 Sonnet2024.10 | 41.5 | 16.3 | 26.5 | |
| ManualLLM=Claude Instant, Few-shot Setting=3-shot (ICL with example selection)2024.10 | 40.4 | 14.8 | 24.8 | |
| ManualLLM=Claude 3 Sonnet, Few-shot Setting=3-shot (ICL with example selection)2024.10 | 40.3 | 15.4 | 25.2 | |
| CriSPOLLM=Claude Instant, Few-shot Setting=0-shot2024.10 | 40.1 | 15.7 | 26.1 | |
| OPROLLM=Claude 3 Sonnet, Few-shot Setting=0-shot2024.10 | 39.7 | 15.1 | 5.1 | |
| OPROLLM=Claude Instant, Few-shot Setting=0-shot2024.10 | 39.5 | 14.3 | 24.5 | |
| OPROLLM=Llama 3 8B, Few-shot Setting=0-shot, Optimizer=Claude 3 Sonnet2024.10 | 39.1 | 15.2 | 24.6 | |
| ManualLLM=Claude 3 Sonnet, Few-shot Setting=0-shot2024.10 | 38.8 | 14.4 | 24 | |
| CriSPOLLM=Mistral 7B, Few-shot Setting=0-shot2024.10 | 38.5 | 14.3 | 23.9 | |
| CriSPOLLM=Mistral 7B, Few-shot Setting=3-shot (ICL with example selection)2024.10 | 38.5 | 14.3 | 24.1 | |
| ManualLLM=Llama 3 8B, Few-shot Setting=0-shot2024.10 | 37.9 | 14.4 | 23.8 | |
| ManualLLM=Claude Instant, Few-shot Setting=0-shot2024.10 | 37.5 | 12.5 | 22.6 | |
| OPROLLM=Mistral 7B, Few-shot Setting=0-shot2024.10 | 36.5 | 14.4 | 23 | |
| ManualLLM=Mistral 7B, Few-shot Setting=0-shot2024.10 | 30.9 | 11 | 20.4 | |
| ManualLLM=Mistral 7B, Few-shot Setting=3-shot (ICL with example selection)2024.10 | 30.7 | 10.6 | 20.1 |