Summarization on GovReport (test)
0.555ROUGE-1GloSA-sum
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GloSA-sumEncoder=GloSA-sum2026.02 | 0.555 | 0.26 | 0.51 | 0.91 | 0.81 | |
| FullKVModel=Gemma, KV-cache=unpruned2026.04 | 0.3761 | 0.1357 | 0.1922 | 0.8725 | — | |
| Claude-3 SumEncoder=Claude-3 Sum2026.02 | 0.3488 | 0.1354 | 0.3166 | 0.855 | 0.793 | |
| GPT-4 Prompt-SumEncoder=GPT-4 Prompt-Sum2026.02 | 0.3321 | 0.1267 | 0.3014 | 0.847 | 0.785 | |
| MGAModel=Gemma, Framework=DepthKV, Allocation Strategy=Maximum Gradient Alignment2026.04 | 0.2843 | 0.0705 | 0.1636 | 0.7024 | — | |
| w/ V (ℓ1)Model=Gemma, Strategy=Uniform Pruning2026.04 | 0.2703 | 0.0589 | 0.1574 | 0.6153 | — | |
| w/o VModel=Gemma, Strategy=Uniform Pruning2026.04 | 0.2676 | 0.0598 | 0.1568 | 0.6205 | — | |
| w/ V (ℓ2)Model=Gemma, Strategy=Uniform Pruning2026.04 | 0.2675 | 0.0598 | 0.1564 | 0.6165 | — | |
| MAML-6LModel=Gemma, Framework=DepthKV, Allocation Strategy=Multi-Layer Alignment (6 layers)2026.04 | 0.2524 | 0.0609 | 0.1447 | 0.6575 | — | |
| MLMA-2LModel=Gemma, Framework=DepthKV, Allocation Strategy=Multi-Layer Maximum Alignment (2 layers)2026.04 | 0.2413 | 0.0596 | 0.1406 | 0.659 | — | |
| MLPModel=Gemma, Framework=DepthKV, Allocation Strategy=MLP2026.04 | 0.2324 | 0.0541 | 0.139 | 0.6182 | — | |
| MLMA-4LModel=Gemma, Framework=DepthKV, Allocation Strategy=Multi-Layer Maximum Alignment (4 layers)2026.04 | 0.2318 | 0.0554 | 0.136 | 0.6352 | — | |
| FT-LLaMA-3 8BEncoder=FT-LLaMA-3 8B2026.02 | 0.2301 | 0.0872 | 0.2187 | 0.803 | 0.731 |