Summarization on MeetingBank (BERTScore F)
90.8BERTScore FCriSPO
Evaluation Results
| Method | Links | |
|---|---|---|
| CriSPOLLM=Claude 3 Sonnet, Prompt Strategy=Automatic, Shots=3-shot, Example Selection=true2024.10 | 90.8 | |
| CriSPOLLM=Claude Instant, Prompt Strategy=Automatic, Shots=3-shot, Example Selection=true2024.10 | 89.2 | |
| CriSPOLLM=Claude 3 Sonnet, Prompt Strategy=Automatic, Shots=0-shot2024.10 | 88.1 | |
| CriSPOLLM=Llama3 8B, Prompt Strategy=Automatic, Shots=0-shot, Optimizer=Claude 3 Sonnet2024.10 | 87.7 | |
| OPROLLM=Claude 3 Sonnet, Prompt Strategy=Automatic2024.10 | 87.1 | |
| ManualLLM=Claude 3 Sonnet, Shots=3-shot, Example Selection=true2024.10 | 86.9 | |
| CriSPOLLM=Claude Instant, Prompt Strategy=Automatic, Shots=0-shot2024.10 | 86.8 | |
| OPROLLM=Claude Instant, Prompt Strategy=Automatic2024.10 | 86.7 | |
| OPROLLM=Llama3 8B, Prompt Strategy=Automatic, Optimizer=Claude 3 Sonnet2024.10 | 86.7 | |
| CriSPOLLM=Mistral 7B, Prompt Strategy=Automatic, Shots=0-shot2024.10 | 86.2 | |
| ManualLLM=Claude Instant, Shots=3-shot, Example Selection=true2024.10 | 86 | |
| CriSPOLLM=Mistral 7B, Prompt Strategy=Automatic, Shots=3-shot, Example Selection=true2024.10 | 85.9 | |
| OPROLLM=Mistral 7B, Prompt Strategy=Automatic2024.10 | 85.8 | |
| ManualLLM=Claude 3 Sonnet, Shots=0-shot2024.10 | 85.4 | |
| ManualLLM=Llama3 8B, Shots=0-shot2024.10 | 85.4 | |
| ManualLLM=Mistral 7B, Shots=3-shot, Example Selection=true2024.10 | 85.3 | |
| ManualLLM=Claude Instant, Shots=0-shot2024.10 | 85 | |
| ManualLLM=Mistral 7B, Shots=0-shot2024.10 | 84.3 |