Summarization on CNN (AlignScore)
86.4AlignScoreCriSPO
Evaluation results
| Method | Links | |
|---|---|---|
| CriSPOLLM=Mistral 7B, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 86.4 | |
| CriSPOLLM=Llama3 8B, optimizer=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 86.1 | |
| ManualLLM=Claude3 Sonnet, shots=3, example_selection=ICL2024.10 | 86 | |
| OPROLLM=Claude Instant, prompt_engineering=Automatic2024.10 | 85.5 | |
| OPROLLM=Llama3 8B, optimizer=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 85.4 | |
| ManualLLM=Mistral 7B, shots=3, example_selection=ICL2024.10 | 85.2 | |
| ManualLLM=Mistral 7B, shots=02024.10 | 84.9 | |
| OPROLLM=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 84.6 | |
| ManualLLM=Claude3 Sonnet, shots=02024.10 | 84.5 | |
| CriSPOLLM=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 84.5 | |
| OPROLLM=Mistral 7B, prompt_engineering=Automatic2024.10 | 84.5 | |
| CriSPOLLM=Mistral 7B, prompt_engineering=Automatic2024.10 | 84.4 | |
| CriSPOLLM=Claude3 Sonnet, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 83.9 | |
| ManualLLM=Llama3 8B, shots=02024.10 | 83.7 | |
| ManualLLM=Claude Instant, shots=3, example_selection=ICL2024.10 | 83.1 | |
| Reference Summary2024.10 | 78.7 | |
| CriSPOLLM=Claude Instant, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 77.8 | |
| ManualLLM=Claude Instant, shots=02024.10 | 76.1 | |
| CriSPOLLM=Claude Instant, prompt_engineering=Automatic2024.10 | 73.9 |