Summarization on ACI-Bench (AlignScore)
72.5AlignScoreManual
Evaluation Results
| Method | Links | |
|---|---|---|
| ManualLLM=Llama3 8B, shots=02024.10 | 72.5 | |
| ManualLLM=Claude3 Sonnet, shots=02024.10 | 70.2 | |
| OPROLLM=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 69.8 | |
| ManualLLM=Mistral 7B, shots=3, example_selection=ICL2024.10 | 69 | |
| ManualLLM=Mistral 7B, shots=02024.10 | 68 | |
| CriSPOLLM=Mistral 7B, prompt_engineering=Automatic2024.10 | 67.8 | |
| ManualLLM=Claude3 Sonnet, shots=3, example_selection=ICL2024.10 | 67.4 | |
| CriSPOLLM=Mistral 7B, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 67.2 | |
| ManualLLM=Claude Instant, shots=02024.10 | 66.7 | |
| ManualLLM=Claude Instant, shots=3, example_selection=ICL2024.10 | 66.3 | |
| CriSPOLLM=Claude Instant, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 65.6 | |
| OPROLLM=Mistral 7B, prompt_engineering=Automatic2024.10 | 65.6 | |
| CriSPOLLM=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 65 | |
| CriSPOLLM=Claude3 Sonnet, shots=3, example_selection=ICL, prompt_engineering=Automatic2024.10 | 63.8 | |
| CriSPOLLM=Claude Instant, prompt_engineering=Automatic2024.10 | 63.3 | |
| OPROLLM=Claude Instant, prompt_engineering=Automatic2024.10 | 62.3 | |
| CriSPOLLM=Llama3 8B, optimizer=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 62.3 | |
| Reference Summary2024.10 | 61.4 | |
| OPROLLM=Llama3 8B, optimizer=Claude3 Sonnet, prompt_engineering=Automatic2024.10 | 59.4 |