Summarization on SAMSum (AlignScore)
89.5AlignScoreOPRO
Evaluation Results
| Method | Links | |
|---|---|---|
| OPROLLM=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 89.5 | |
| OPROLLM=Llama3 8B, optimizer=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 88.9 | |
| ManualLLM=Llama3 8B, shots=02024.10 | 88.8 | |
| OPROLLM=Mistral 7B, prompt_engineering=Automatic2024.10 | 88.4 | |
| ManualLLM=Claude3 Sonnet, shots=02024.10 | 87.9 | |
| CriSPOLLM=Mistral 7B, prompt_engineering=Automatic2024.10 | 87.7 | |
| CriSPOLLM=Llama3 8B, optimizer=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 87.7 | |
| ManualLLM=Mistral 7B, shots=02024.10 | 87.6 | |
| CriSPOLLM=Mistral 7B, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 87.4 | |
| ManualLLM=Claude3 Sonnet, shots=3, example_selection=ICL2024.10 | 87.2 | |
| CriSPOLLM=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 87 | |
| ManualLLM=Mistral 7B, shots=3, example_selection=ICL2024.10 | 86.8 | |
| ManualLLM=Claude Instant, shots=3, example_selection=ICL2024.10 | 86.6 | |
| ManualLLM=Claude Instant, shots=02024.10 | 85.7 | |
| CriSPOLLM=Claude Instant, prompt_engineering=Automatic2024.10 | 85.3 | |
| OPROLLM=Claude Instant, prompt_engineering=Automatic2024.10 | 84.5 | |
| CriSPOLLM=Claude3 Sonnet, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 84.4 | |
| CriSPOLLM=Claude Instant, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 83.9 | |
| Reference Summary2024.10 | 79.9 |