Summarization on ACI-Bench (test)
41ROUGE-L F1CriSPO
Evaluation Results
| Method | Links | |
|---|---|---|
| CriSPOLLM=Claude3 Sonnet, Shots=32024.10 | 41 | |
| ManualLLM=Claude3 Sonnet, Shots=32024.10 | 38.6 | |
| CriSPOLLM=Claude Instant, Shots=32024.10 | 35.3 | |
| ManualLLM=Claude Instant, Shots=32024.10 | 33.5 | |
| CriSPOLLM=Claude3 Sonnet, Shots=02024.10 | 30.3 | |
| CriSPOLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.10 | 29.9 | |
| OPROLLM=Claude3 Sonnet, Shots=02024.10 | 29.5 | |
| ManualLLM=Claude3 Sonnet, Shots=02024.10 | 29.3 | |
| OPROLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.10 | 29.3 | |
| ManualLLM=Mistral 7B, Shots=32024.10 | 28.1 | |
| ManualLLM=Llama3 8B, Shots=02024.10 | 27.7 | |
| CriSPOLLM=Claude Instant, Shots=02024.10 | 26.8 | |
| CriSPOLLM=Mistral 7B, Shots=32024.10 | 26.2 | |
| ManualLLM=Claude Instant, Shots=02024.10 | 26.1 | |
| CriSPOLLM=Mistral 7B, Shots=02024.10 | 25.6 | |
| OPROLLM=Claude Instant, Shots=02024.10 | 25.5 | |
| ManualLLM=Mistral 7B, Shots=02024.10 | 25.4 | |
| OPROLLM=Mistral 7B, Shots=02024.10 | 25.2 |