Long-Context Summarization on GovReport (test)
16.11ROUGE-1FlashMem
Evaluation results
| Method | Links | |
|---|---|---|
| FlashMemModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=FlashMem2026.01 | 16.11 | |
| FlashMemModel Backbone=Qwen 3 4B Instruct, Inference Strategy=FlashMem2026.01 | 15.52 | |
| CoT-SCModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=CoT-SC2026.01 | 14.41 | |
| MemGenModel Backbone=Qwen 3 4B Instruct, Inference Strategy=MemGen2026.01 | 14.31 | |
| SnapKVModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=SnapKV2026.01 | 14.22 | |
| MemGenModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=MemGen2026.01 | 13.97 | |
| CoT-SCModel Backbone=Qwen 3 4B Instruct, Inference Strategy=CoT-SC2026.01 | 13.87 | |
| SnapKVModel Backbone=Qwen 3 4B Instruct, Inference Strategy=SnapKV2026.01 | 13.74 | |
| VanillaModel Backbone=Qwen 3 4B Instruct, Inference Strategy=Vanilla2026.01 | 13.65 | |
| VanillaModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=Vanilla2026.01 | 13.61 |