Summarization on CNN
17.3ROUGE-2 F-ScoreCriSPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| CriSPOLLM=Claude3 Sonnet, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot2024.10 | 17.3 | — | |
| CriSPOLLM=Claude Instant, Strategy=Automatic Prompt Engineering, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 17 | — | |
| CriSPOLLM=Claude3 Sonnet, Strategy=Automatic Prompt Engineering, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 16.3 | — | |
| CriSPOLLM=Llama3 8B, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot, Optimizer=Claude3 Sonnet2024.10 | 16.3 | — | |
| CriSPOLLM=Claude Instant, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot2024.10 | 15.7 | — | |
| ManualLLM=Claude3 Sonnet, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 15.4 | — | |
| OPROLLM=Llama3 8B, Strategy=Automatic Prompt Engineering, Optimizer=Claude3 Sonnet2024.10 | 15.2 | — | |
| OPROLLM=Claude3 Sonnet, Strategy=Automatic Prompt Engineering2024.10 | 15.1 | — | |
| ManualLLM=Claude Instant, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 14.8 | — | |
| ManualLLM=Claude3 Sonnet, Evaluation Protocol=0-shot2024.10 | 14.4 | — | |
| OPROLLM=Mistral 7B, Strategy=Automatic Prompt Engineering2024.10 | 14.4 | — | |
| ManualLLM=Llama3 8B, Evaluation Protocol=0-shot2024.10 | 14.4 | — | |
| OPROLLM=Claude Instant, Strategy=Automatic Prompt Engineering2024.10 | 14.3 | — | |
| CriSPOLLM=Mistral 7B, Strategy=Automatic Prompt Engineering, Evaluation Protocol=0-shot2024.10 | 14.3 | — | |
| CriSPOLLM=Mistral 7B, Strategy=Automatic Prompt Engineering, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 14.3 | — | |
| ManualLLM=Claude Instant, Evaluation Protocol=0-shot2024.10 | 12.5 | — | |
| ManualLLM=Mistral 7B, Evaluation Protocol=0-shot2024.10 | 11 | — | |
| ManualLLM=Mistral 7B, Evaluation Protocol=3-shot (ICL with example selection)2024.10 | 10.6 | — | |
| BEST-ON-VALBackbone=Falcon2024.12 | — | 8.4 | |
| BEST-ON-VALBackbone=Llama-22024.12 | — | 11.8 | |
| RANDOMBackbone=Falcon2024.12 | — | 7.1 | |
| RANDOMBackbone=Llama-22024.12 | — | 7.3 | |
| SMOOTHIE-GLOBALBackbone=Falcon2024.12 | — | 7.9 | |
| SMOOTHIE-GLOBALBackbone=Llama-22024.12 | — | 6.9 | |
| SMOOTHIE-LOCALBackbone=Falcon2024.12 | — | 8 | |
| SMOOTHIE-LOCALBackbone=Llama-22024.12 | — | 9.5 |