List your company
ResearchFormerly WizWand

Mathematical Reasoning on GSM8K (test) (Accuracy, 95% CI)

96.5Accuracy (GSM8K)Seed
51.67663.31374.9586.587May 25, 2026May 31, 2026Jun 6, 2026Jun 12, 2026Jun 18, 2026Jun 24, 2026Jun 30, 2026
Updated 4mo ago

Evaluation results

MethodLinks
SeedBootstrap resampling=N=10002026.0596.5——
SPEARBootstrap resampling=N=10002026.0596.50.955—
GEPABootstrap resampling=N=10002026.05960.948—
TextGradBootstrap resampling=N=10002026.0595.60.945—
Self-ConsistencyBackbone=Qwen2.5-7B-Instruct, Method Category=Non-RL Baselines2026.0592.4—62.42
CoTBackbone=Qwen2.5-7B-Instruct, Method Category=Non-RL Baselines2026.0591.6—67.83
Self-ConsistencyBackbone=Phi-3 Mini 4K Instruct, Method Category=Non-RL Baselines2026.0590.37—52.32
TRACERBackbone=Qwen2.5-7B-Instruct, Method Category=Non-RL/RL Baseline Representative2026.0589.01—61.85
Sparse MADBackbone=Qwen2.5-7B-Instruct, Method Category=Non-RL Baselines2026.0588.78—55.8
MoABackbone=Qwen2.5-7B-Instruct, Method Category=Non-RL Baselines2026.0588.7—53.39
MADBackbone=Qwen2.5-7B-Instruct, Method Category=Non-RL Baselines2026.0588.62—56.11
Single-Agent GRPOBackbone=Qwen2.5-7B-Instruct, Method Category=RL Baselines2026.0588.25—46.49
TRACERBackbone=Phi-3 Mini 4K Instruct, Method Category=Non-RL/RL Baseline Representative2026.0588.02—50.27
Single-Agent GSPOBackbone=Phi-3 Mini 4K Instruct, Method Category=RL Baselines2026.0587.95—45.72
Single-Agent GSPOBackbone=Qwen2.5-7B-Instruct, Method Category=RL Baselines2026.0587.41—45.63
Single-Agent GRPOBackbone=Phi-3 Mini 4K Instruct, Method Category=RL Baselines2026.0586.81—46.63
FullKVBackbone=QWEN2.5-14B-INSTRUCT, 50-shot prompting=true, KV-cache budget=uncompressed2026.0686.7——
Sparse MADBackbone=Phi-3 Mini 4K Instruct, Method Category=Non-RL Baselines2026.0584.69—51.13
SEKVBackbone=QWEN2.5-14B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0684.5——
MAPoRLBackbone=Phi-3 Mini 4K Instruct, Method Category=RL Baselines2026.0584.15—48.93
MADBackbone=Phi-3 Mini 4K Instruct, Method Category=Non-RL Baselines2026.0584.08—51.1
MAPoRLBackbone=Qwen2.5-7B-Instruct, Method Category=RL Baselines2026.0583.85—38.19
CoTBackbone=Phi-3 Mini 4K Instruct, Method Category=Non-RL Baselines2026.0582.5—52.2
FullKVBackbone=LLAMA-3.1-8B-INSTRUCT, 50-shot prompting=true, KV-cache budget=uncompressed2026.0682.4——
SentenceKVBackbone=QWEN2.5-14B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0682.1——
ChunkKVBackbone=QWEN2.5-14B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0680.7——
MoABackbone=Phi-3 Mini 4K Instruct, Method Category=Non-RL Baselines2026.0580.44—47.26
PyramidKVBackbone=QWEN2.5-14B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0679.4——
SEKVBackbone=LLAMA-3.1-8B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0679.3——
SnapKVBackbone=QWEN2.5-14B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0677.9——
SentenceKVBackbone=LLAMA-3.1-8B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0676.8——
ChunkKVBackbone=LLAMA-3.1-8B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0674.9——
FullKVBackbone=MISTRAL-7B-INSTRUCT, 50-shot prompting=true, KV-cache budget=uncompressed2026.0674.6——
MAGRPOBackbone=Phi-3 Mini 4K Instruct, Method Category=RL Baselines2026.0572.52—38.52
SEKVBackbone=MISTRAL-7B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0672.5——
SentenceKVBackbone=MISTRAL-7B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0670.4——
PyramidKVBackbone=LLAMA-3.1-8B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0670.3——
MAGRPOBackbone=Qwen2.5-7B-Instruct, Method Category=RL Baselines2026.0569.6—34.73
FullKVBackbone=LLAMA-3-8B, 50-shot prompting=true, KV-cache budget=uncompressed2026.0669.4——
ChunkKVBackbone=MISTRAL-7B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0669.3——
SnapKVBackbone=LLAMA-3.1-8B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0668.2——
PyramidKVBackbone=MISTRAL-7B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0668.1——
SEKVBackbone=LLAMA-3-8B, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0667.8——
SnapKVBackbone=MISTRAL-7B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0666.7——
SentenceKVBackbone=LLAMA-3-8B, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0665.4——
ChunkKVBackbone=LLAMA-3-8B, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0664.6——
PyramidKVBackbone=LLAMA-3-8B, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0663.2——
SnapKVBackbone=LLAMA-3-8B, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0661.7——
FullKVBackbone=LLAMA-3.2-3B-INSTRUCT, 50-shot prompting=true, KV-cache budget=uncompressed2026.0660.8——
SEKVBackbone=LLAMA-3.2-3B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0658.4——
SentenceKVBackbone=LLAMA-3.2-3B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0656.1——
ChunkKVBackbone=LLAMA-3.2-3B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0655.6——
PyramidKVBackbone=LLAMA-3.2-3B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0654.2——
SnapKVBackbone=LLAMA-3.2-3B-INSTRUCT, 50-shot prompting=true, KV-cache budget=10% GPU-resident2026.0653.4——