Summarization on CNN (BERTScore F)
87.8BERTScore FCriSPO
Evaluation Results
| Method | Links | |
|---|---|---|
| CriSPOLLM=Claude 3 Sonnet, Prompt Strategy=Automatic, Shots=0-shot2024.10 | 87.8 | |
| CriSPOLLM=Claude 3 Sonnet, Prompt Strategy=Automatic, Shots=3-shot, Example Selection=true2024.10 | 87.8 | |
| CriSPOLLM=Claude Instant, Prompt Strategy=Automatic, Shots=3-shot, Example Selection=true2024.10 | 87.7 | |
| ManualLLM=Claude 3 Sonnet, Shots=3-shot, Example Selection=true2024.10 | 87.7 | |
| ManualLLM=Claude Instant, Shots=3-shot, Example Selection=true2024.10 | 87.6 | |
| CriSPOLLM=Llama3 8B, Prompt Strategy=Automatic, Shots=0-shot, Optimizer=Claude 3 Sonnet2024.10 | 87.6 | |
| OPROLLM=Claude Instant, Prompt Strategy=Automatic2024.10 | 87.5 | |
| OPROLLM=Claude 3 Sonnet, Prompt Strategy=Automatic2024.10 | 87.5 | |
| ManualLLM=Claude 3 Sonnet, Shots=0-shot2024.10 | 87.4 | |
| OPROLLM=Llama3 8B, Prompt Strategy=Automatic, Optimizer=Claude 3 Sonnet2024.10 | 87.4 | |
| CriSPOLLM=Mistral 7B, Prompt Strategy=Automatic, Shots=0-shot2024.10 | 87.3 | |
| CriSPOLLM=Mistral 7B, Prompt Strategy=Automatic, Shots=3-shot, Example Selection=true2024.10 | 87.3 | |
| CriSPOLLM=Claude Instant, Prompt Strategy=Automatic, Shots=0-shot2024.10 | 87.2 | |
| ManualLLM=Llama3 8B, Shots=0-shot2024.10 | 87.2 | |
| ManualLLM=Claude Instant, Shots=0-shot2024.10 | 87 | |
| OPROLLM=Mistral 7B, Prompt Strategy=Automatic2024.10 | 87 | |
| ManualLLM=Mistral 7B, Shots=3-shot, Example Selection=true2024.10 | 85.8 | |
| ManualLLM=Mistral 7B, Shots=0-shot2024.10 | 85.6 |