List your company
ResearchFormerly WizWand

Summarization on GovReport (ROUGE)

46ROUGE ScoreSMD
23.22429.13735.0540.963Feb 20, 2025May 4, 2025Jul 17, 2025Sep 28, 2025Dec 11, 2025Feb 22, 2026May 7, 2026
Updated 5mo ago

Evaluation results

MethodLinks
SMDModel=Qwen2.5-7B, Rollout Method=SMD, Toks. saving=39.4%2026.0546
GRPO DenseModel=Qwen2.5-7B, Rollout Method=Dense2026.0545.2
Sparse-RLModel=Qwen2.5-7B, Rollout Method=Sparse-RL, Toks. saving=39.4%2026.0544.1
GRPO DenseModel=Qwen3-4B-Instruct, Rollout Method=Dense2026.0542.5
Dense BaselineKV cache=100%2026.0542.5
SMDModel=Qwen3-4B-Instruct, Rollout Method=SMD, Toks. saving=50.0%2026.0542.4
SMD SnapKVKV cache=50% drop2026.0542.4
BaseModel=Qwen2.5-7B, Rollout Method=–2026.0541.5
SMDModel=Qwen2.5-3B, Rollout Method=SMD, Toks. saving=42.0%2026.0541
GRPO DenseModel=Qwen2.5-3B, Rollout Method=Dense2026.0540.2
Sparse-RLModel=Qwen3-4B-Instruct, Rollout Method=Sparse-RL, Toks. saving=50.0%2026.0539.8
Sparse-RLKV cache=50% drop2026.0539.8
Sparse-RLModel=Qwen2.5-3B, Rollout Method=Sparse-RL, Toks. saving=42.0%2026.0539.5
GRPO w/ SnapKVModel=Qwen2.5-7B, Rollout Method=w/ SnapKV2026.0538.6
BaseModel=Qwen3-4B-Instruct, Rollout Method=–2026.0538
BaseModel=Qwen2.5-3B, Rollout Method=–2026.0536.4
GRPO w/ SnapKVModel=Qwen3-4B-Instruct, Rollout Method=w/ SnapKV2026.0535.2
SMDModel=Qwen2.5-1.5B, Rollout Method=SMD, Toks. saving=43.3%2026.0534.8
GRPO w/ SnapKVModel=Qwen2.5-3B, Rollout Method=w/ SnapKV2026.0534.8
GRPO DenseModel=Qwen2.5-1.5B, Rollout Method=Dense2026.0534.2
Sparse-RLModel=Qwen2.5-1.5B, Rollout Method=Sparse-RL, Toks. saving=43.3%2026.0533.5
LIFTTarget Model=Llama-3-8B-Instruct, Synthetic Task Generator=Qwen2.5-72B-Instruct2025.0230.9
GRPO w/ SnapKVModel=Qwen2.5-1.5B, Rollout Method=w/ SnapKV2026.0530
ICLEvaluation Mode=Truncated In-Context Learning, Target Model=Llama-3-8B-Instruct2025.0229.1
SMDModel=Llama-3.2-1B-Instruct, Rollout Method=SMD, Toks. saving=45.0%2026.0528.8
GRPO DenseModel=Llama-3.2-1B-Instruct, Rollout Method=Dense2026.0528.5
BaseModel=Qwen2.5-1.5B, Rollout Method=–2026.0528.1
Sparse-RLModel=Llama-3.2-1B-Instruct, Rollout Method=Sparse-RL, Toks. saving=45.0%2026.0526.5
BaseModel=Llama-3.2-1B-Instruct, Rollout Method=–2026.0524.2
GRPO w/ SnapKVModel=Llama-3.2-1B-Instruct, Rollout Method=w/ SnapKV2026.0524.1