Summarization on ArXiv (test) (ROUGE, BS)
85.03BS ScoreSelf-Consistency (SC)
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Self-Consistency (SC)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 85.03 | 23.97 | — | — | — | |
| Iterative Refine (IR)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 85.03 | 23.93 | — | — | — | |
| Extract-to-Abstract (E2A)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 84.83 | 22.9 | — | — | — | |
| Chain-of-Thought (CoT)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 84.35 | 23.64 | — | — | — | |
| Question-Answer Guided (QAG)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 84.3 | 22.41 | — | — | — | |
| o1Backbone=Large Reasoning Model, Shot-setting=2-shot2025.12 | 84.14 | 20.88 | — | — | — | |
| Cited Summarization (Cite)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 84.13 | 23.55 | — | — | — | |
| VanillaBackbone=GPT-4.1, Shot-setting=2-shot2025.12 | 83.83 | 23.29 | — | — | — | |
| Plan-then-Write (Plan)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 83.15 | 19.79 | — | — | — | |
| GPT-5Backbone=Large Reasoning Model, Shot-setting=2-shot2025.12 | 82.11 | 17.22 | — | — | — | |
| o3Backbone=Large Reasoning Model, Shot-setting=2-shot2025.12 | 81.44 | 16.3 | — | — | — | |
| Decomposition (Deco)Backbone=GPT-4.1, Shot-setting=2-shot2025.12 | 79.8 | 19.5 | — | — | — | |
| FullKVModel=Gemma, KV-cache=unpruned2026.04 | 79.42 | — | 39.09 | 13.84 | 22.57 | |
| MGAModel=Gemma, Framework=DepthKV, Allocation Strategy=Maximum Gradient Alignment2026.04 | 61.98 | — | 29.75 | 6.92 | 18.59 | |
| MLMA-2LModel=Gemma, Framework=DepthKV, Allocation Strategy=Multi-Layer Maximum Alignment (2 layers)2026.04 | 61.19 | — | 29.54 | 6.55 | 18.52 | |
| MAML-6LModel=Gemma, Framework=DepthKV, Allocation Strategy=Multi-Layer Alignment (6 layers)2026.04 | 58.95 | — | 28.92 | 6.12 | 17.97 | |
| MLPModel=Gemma, Framework=DepthKV, Allocation Strategy=MLP2026.04 | 57.96 | — | 28.47 | 6.01 | 17.99 | |
| MLMA-4LModel=Gemma, Framework=DepthKV, Allocation Strategy=Multi-Layer Maximum Alignment (4 layers)2026.04 | 57.57 | — | 28.26 | 5.96 | 17.7 | |
| w/o VModel=Gemma, Strategy=Uniform Pruning2026.04 | 55.09 | — | 26.75 | 5.5 | 17.24 | |
| w/ V (ℓ2)Model=Gemma, Strategy=Uniform Pruning2026.04 | 54 | — | 26.63 | 5.42 | 17.06 | |
| w/ V (ℓ1)Model=Gemma, Strategy=Uniform Pruning2026.04 | 53.9 | — | 26.84 | 5.58 | 17.21 |