List your company
ResearchFormerly WizWand

Multi-hop Question Answering on HotpotQA (F1 Score)

56.2F1 ScoreSMD
28.74435.8724350.128May 7, 2026
Updated 6mo ago

Evaluation results

MethodLinks
SMDModel=Qwen2.5-7B, Rollout Method=SMD, Toks. saving=39.4%2026.0556.2
GRPO DenseModel=Qwen2.5-7B, Rollout Method=Dense2026.0555.8
Sparse-RLModel=Qwen2.5-7B, Rollout Method=Sparse-RL, Toks. saving=39.4%2026.0554.5
SMDModel=Qwen3-4B-Instruct, Rollout Method=SMD, Toks. saving=50.0%2026.0552.6
GRPO DenseModel=Qwen3-4B-Instruct, Rollout Method=Dense2026.0552.4
SMDModel=Qwen2.5-3B, Rollout Method=SMD, Toks. saving=42.0%2026.0551.5
BaseModel=Qwen2.5-7B, Rollout Method=–2026.0551.2
GRPO DenseModel=Qwen2.5-3B, Rollout Method=Dense2026.0550.6
Sparse-RLModel=Qwen2.5-3B, Rollout Method=Sparse-RL, Toks. saving=42.0%2026.0550
Sparse-RLModel=Qwen3-4B-Instruct, Rollout Method=Sparse-RL, Toks. saving=50.0%2026.0548.6
GRPO w/ SnapKVModel=Qwen2.5-7B, Rollout Method=w/ SnapKV2026.0548
GRPO w/ SnapKVModel=Qwen2.5-3B, Rollout Method=w/ SnapKV2026.0546.5
BaseModel=Qwen3-4B-Instruct, Rollout Method=–2026.0545.2
BaseModel=Qwen2.5-3B, Rollout Method=–2026.0545.1
SMDModel=Qwen2.5-1.5B, Rollout Method=SMD, Toks. saving=43.3%2026.0541.6
GRPO w/ SnapKVModel=Qwen3-4B-Instruct, Rollout Method=w/ SnapKV2026.0541.5
GRPO DenseModel=Qwen2.5-1.5B, Rollout Method=Dense2026.0541
Sparse-RLModel=Qwen2.5-1.5B, Rollout Method=Sparse-RL, Toks. saving=43.3%2026.0540.2
GRPO w/ SnapKVModel=Qwen2.5-1.5B, Rollout Method=w/ SnapKV2026.0536.5
SMDModel=Llama-3.2-1B-Instruct, Rollout Method=SMD, Toks. saving=45.0%2026.0536.1
GRPO DenseModel=Llama-3.2-1B-Instruct, Rollout Method=Dense2026.0535.4
BaseModel=Qwen2.5-1.5B, Rollout Method=–2026.0533.5
Sparse-RLModel=Llama-3.2-1B-Instruct, Rollout Method=Sparse-RL, Toks. saving=45.0%2026.0533.2
BaseModel=Llama-3.2-1B-Instruct, Rollout Method=–2026.0530.1
GRPO w/ SnapKVModel=Llama-3.2-1B-Instruct, Rollout Method=w/ SnapKV2026.0529.8