Multi-hop Question Answering on HotpotQA (F1 Score)
56.2F1 ScoreSMD
Evaluation Results
| Method | Links | |
|---|---|---|
| SMDModel=Qwen2.5-7B, Rollout Method=SMD, Toks. saving=39.4%2026.05 | 56.2 | |
| GRPO DenseModel=Qwen2.5-7B, Rollout Method=Dense2026.05 | 55.8 | |
| Sparse-RLModel=Qwen2.5-7B, Rollout Method=Sparse-RL, Toks. saving=39.4%2026.05 | 54.5 | |
| SMDModel=Qwen3-4B-Instruct, Rollout Method=SMD, Toks. saving=50.0%2026.05 | 52.6 | |
| GRPO DenseModel=Qwen3-4B-Instruct, Rollout Method=Dense2026.05 | 52.4 | |
| SMDModel=Qwen2.5-3B, Rollout Method=SMD, Toks. saving=42.0%2026.05 | 51.5 | |
| BaseModel=Qwen2.5-7B, Rollout Method=–2026.05 | 51.2 | |
| GRPO DenseModel=Qwen2.5-3B, Rollout Method=Dense2026.05 | 50.6 | |
| Sparse-RLModel=Qwen2.5-3B, Rollout Method=Sparse-RL, Toks. saving=42.0%2026.05 | 50 | |
| Sparse-RLModel=Qwen3-4B-Instruct, Rollout Method=Sparse-RL, Toks. saving=50.0%2026.05 | 48.6 | |
| GRPO w/ SnapKVModel=Qwen2.5-7B, Rollout Method=w/ SnapKV2026.05 | 48 | |
| GRPO w/ SnapKVModel=Qwen2.5-3B, Rollout Method=w/ SnapKV2026.05 | 46.5 | |
| BaseModel=Qwen3-4B-Instruct, Rollout Method=–2026.05 | 45.2 | |
| BaseModel=Qwen2.5-3B, Rollout Method=–2026.05 | 45.1 | |
| SMDModel=Qwen2.5-1.5B, Rollout Method=SMD, Toks. saving=43.3%2026.05 | 41.6 | |
| GRPO w/ SnapKVModel=Qwen3-4B-Instruct, Rollout Method=w/ SnapKV2026.05 | 41.5 | |
| GRPO DenseModel=Qwen2.5-1.5B, Rollout Method=Dense2026.05 | 41 | |
| Sparse-RLModel=Qwen2.5-1.5B, Rollout Method=Sparse-RL, Toks. saving=43.3%2026.05 | 40.2 | |
| GRPO w/ SnapKVModel=Qwen2.5-1.5B, Rollout Method=w/ SnapKV2026.05 | 36.5 | |
| SMDModel=Llama-3.2-1B-Instruct, Rollout Method=SMD, Toks. saving=45.0%2026.05 | 36.1 | |
| GRPO DenseModel=Llama-3.2-1B-Instruct, Rollout Method=Dense2026.05 | 35.4 | |
| BaseModel=Qwen2.5-1.5B, Rollout Method=–2026.05 | 33.5 | |
| Sparse-RLModel=Llama-3.2-1B-Instruct, Rollout Method=Sparse-RL, Toks. saving=45.0%2026.05 | 33.2 | |
| BaseModel=Llama-3.2-1B-Instruct, Rollout Method=–2026.05 | 30.1 | |
| GRPO w/ SnapKVModel=Llama-3.2-1B-Instruct, Rollout Method=w/ SnapKV2026.05 | 29.8 |