Abstractive Summarization on MeetingBank (test)
54.1ROUGE-L F-scoreCriSPO
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| CriSPOLLM=Claude3 Sonnet, Shots=32024.10 | 54.1 | — | — | — | |
| CriSPOLLM=Claude Instant, Shots=32024.10 | 44.4 | — | — | — | |
| CriSPOLLM=Claude3 Sonnet, Shots=02024.10 | 40.9 | — | — | — | |
| CriSPOLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.10 | 36.8 | — | — | — | |
| CriSPOLLM=Claude Instant, Shots=02024.10 | 33.1 | — | — | — | |
| OPROLLM=Claude3 Sonnet, Shots=02024.10 | 32 | — | — | — | |
| OPROLLM=Llama3 8B, Shots=0, Optimizer=Claude3 Sonnet2024.10 | 31.5 | — | — | — | |
| OPROLLM=Claude Instant, Shots=02024.10 | 29.7 | — | — | — | |
| ManualLLM=Claude3 Sonnet, Shots=32024.10 | 29.5 | — | — | — | |
| CriSPOLLM=Mistral 7B, Shots=02024.10 | 29.3 | — | — | — | |
| CriSPOLLM=Mistral 7B, Shots=32024.10 | 26.1 | — | — | — | |
| ManualLLM=Claude Instant, Shots=32024.10 | 25.5 | — | — | — | |
| OPROLLM=Mistral 7B, Shots=02024.10 | 24.2 | — | — | — | |
| ManualLLM=Mistral 7B, Shots=32024.10 | 22.7 | — | — | — | |
| ManualLLM=Llama3 8B, Shots=02024.10 | 22.6 | — | — | — | |
| ManualLLM=Claude3 Sonnet, Shots=02024.10 | 22.3 | — | — | — | |
| ManualLLM=Claude Instant, Shots=02024.10 | 20.5 | — | — | — | |
| ManualLLM=Mistral 7B, Shots=02024.10 | 18.5 | — | — | — | |
| 0-shot SOTAMode=Zero-shot2024.10 | — | 36.4 | 26.8 | — | |
| Claude InstantBackbone=Claude-Instant-12024.10 | — | 32.2 | 21.8 | 43.4 | |
| Claude Instant + 2-stageBackbone=Claude-Instant-12024.10 | — | 30.8 | 20.7 | 43.8 | |
| Claude Instant + GP-SigExtBackbone=Claude-Instant-12024.10 | — | 36.3 | 25.7 | 53.1 | |
| Claude Instant + SigExtBackbone=Claude-Instant-12024.10 | — | 42.3 | 31.9 | 60.5 | |
| Mistral-7BBackbone=Mistral-7B-Instruct2024.10 | — | 34.4 | 25.2 | 50.3 | |
| Mistral-7B + 2-stageBackbone=Mistral-7B-Instruct2024.10 | — | 32 | 23.5 | 52 | |
| Mistral-7B + GP-SigExtBackbone=Mistral-7B-Instruct2024.10 | — | 34.1 | 24.7 | 54.8 | |
| Mistral-7B + SigExtBackbone=Mistral-7B-Instruct2024.10 | — | 37 | 27.2 | 58.7 |