Summarization on GovReport (ROUGE)
46ROUGE ScoreSMD
Evaluation Results
| Method | Links | |
|---|---|---|
| SMDModel=Qwen2.5-7B, Rollout Method=SMD, Toks. saving=39.4%2026.05 | 46 | |
| GRPO DenseModel=Qwen2.5-7B, Rollout Method=Dense2026.05 | 45.2 | |
| Sparse-RLModel=Qwen2.5-7B, Rollout Method=Sparse-RL, Toks. saving=39.4%2026.05 | 44.1 | |
| GRPO DenseModel=Qwen3-4B-Instruct, Rollout Method=Dense2026.05 | 42.5 | |
| Dense BaselineKV cache=100%2026.05 | 42.5 | |
| SMDModel=Qwen3-4B-Instruct, Rollout Method=SMD, Toks. saving=50.0%2026.05 | 42.4 | |
| SMD SnapKVKV cache=50% drop2026.05 | 42.4 | |
| BaseModel=Qwen2.5-7B, Rollout Method=–2026.05 | 41.5 | |
| SMDModel=Qwen2.5-3B, Rollout Method=SMD, Toks. saving=42.0%2026.05 | 41 | |
| GRPO DenseModel=Qwen2.5-3B, Rollout Method=Dense2026.05 | 40.2 | |
| Sparse-RLModel=Qwen3-4B-Instruct, Rollout Method=Sparse-RL, Toks. saving=50.0%2026.05 | 39.8 | |
| Sparse-RLKV cache=50% drop2026.05 | 39.8 | |
| Sparse-RLModel=Qwen2.5-3B, Rollout Method=Sparse-RL, Toks. saving=42.0%2026.05 | 39.5 | |
| GRPO w/ SnapKVModel=Qwen2.5-7B, Rollout Method=w/ SnapKV2026.05 | 38.6 | |
| BaseModel=Qwen3-4B-Instruct, Rollout Method=–2026.05 | 38 | |
| BaseModel=Qwen2.5-3B, Rollout Method=–2026.05 | 36.4 | |
| GRPO w/ SnapKVModel=Qwen3-4B-Instruct, Rollout Method=w/ SnapKV2026.05 | 35.2 | |
| SMDModel=Qwen2.5-1.5B, Rollout Method=SMD, Toks. saving=43.3%2026.05 | 34.8 | |
| GRPO w/ SnapKVModel=Qwen2.5-3B, Rollout Method=w/ SnapKV2026.05 | 34.8 | |
| GRPO DenseModel=Qwen2.5-1.5B, Rollout Method=Dense2026.05 | 34.2 | |
| Sparse-RLModel=Qwen2.5-1.5B, Rollout Method=Sparse-RL, Toks. saving=43.3%2026.05 | 33.5 | |
| LIFTTarget Model=Llama-3-8B-Instruct, Synthetic Task Generator=Qwen2.5-72B-Instruct2025.02 | 30.9 | |
| GRPO w/ SnapKVModel=Qwen2.5-1.5B, Rollout Method=w/ SnapKV2026.05 | 30 | |
| ICLEvaluation Mode=Truncated In-Context Learning, Target Model=Llama-3-8B-Instruct2025.02 | 29.1 | |
| SMDModel=Llama-3.2-1B-Instruct, Rollout Method=SMD, Toks. saving=45.0%2026.05 | 28.8 | |
| GRPO DenseModel=Llama-3.2-1B-Instruct, Rollout Method=Dense2026.05 | 28.5 | |
| BaseModel=Qwen2.5-1.5B, Rollout Method=–2026.05 | 28.1 | |
| Sparse-RLModel=Llama-3.2-1B-Instruct, Rollout Method=Sparse-RL, Toks. saving=45.0%2026.05 | 26.5 | |
| BaseModel=Llama-3.2-1B-Instruct, Rollout Method=–2026.05 | 24.2 | |
| GRPO w/ SnapKVModel=Llama-3.2-1B-Instruct, Rollout Method=w/ SnapKV2026.05 | 24.1 |