Summarization on BookSum Trun. latest (test)
16.68Avg ROUGECoA
Evaluation Results
| Method | Links | |
|---|---|---|
| CoABackbone=Claude-3-Opus, Context Window=8k2024.06 | 16.68 | |
| CoABackbone=Claude-3-Sonnet, Context Window=8k2024.06 | 15.05 | |
| CoABackbone=Claude-3-Haiku, Context Window=8k2024.06 | 13.73 | |
| VanillaBackbone=Claude-3-Opus, Context Window=200k2024.06 | 13.67 | |
| VanillaBackbone=Claude-3-Sonnet, Context Window=200k2024.06 | 11.9 | |
| VanillaBackbone=Claude-3-Haiku, Context Window=200k2024.06 | 11.7 |