List your company
ResearchFormerly WizWand

Mathematical Reasoning on DeepMath hard (held-out)

73Pass@1Agon
2134.54861.5Jul 8, 2026
Updated 4mo ago

Evaluation results

MethodLinks
AgonModel=Gemma-4-E4B, Protocol=single training run, cascade protocol2026.077315—
AgonModel=Qwen3-4B, Protocol=single training run, cascade protocol2026.077112—
AgonModel=Qwen3-1.7B, Protocol=single training run, cascade protocol2026.077024—
AgonModel=Qwen3.5-2B, Protocol=single training run, cascade protocol2026.077020—
AgonModel=Qwen3-0.6B, Protocol=single training run, cascade protocol2026.076131—
Agon (competition + exchange)Model=Qwen3-0.6B, Inference Passes=two sequential generation passes, Training Protocol=Competition + Exchange2026.0761—3,500
Vanilla GRPOModel=Qwen3-4B, Protocol=single training run, cascade protocol2026.0759——
Vanilla GRPOModel=Gemma-4-E4B, Protocol=single training run, cascade protocol2026.0758——
Zero-shotModel=Qwen3-4B, Protocol=single evaluation2026.0752——
Vanilla GRPOModel=Qwen3.5-2B, Protocol=single training run, cascade protocol2026.0750——
Zero-shotModel=Gemma-4-E4B, Protocol=single evaluation2026.0750——
Vanilla GRPOModel=Qwen3-1.7B, Protocol=single training run, cascade protocol2026.0746——
Cooperative exchangeModel=Qwen3-0.6B, Inference Passes=two sequential generation passes, Training Protocol=Cooperative2026.0746—5,100
Zero-shotModel=Qwen3.5-2B, Protocol=single evaluation2026.0744——
Zero-shotModel=Qwen3-1.7B, Protocol=single evaluation2026.0738——
GRPO two-pass self-cascade (control)Model=Qwen3-0.6B, Inference Passes=two sequential generation passes, Training Protocol=GRPO2026.0735—8,000
MoA (no train)Model=Qwen3-0.6B, Inference Passes=single evaluation, Training=None2026.0734—6,900
Self-refinement (control)Model=Qwen3-0.6B, Inference Passes=two sequential generation passes, Training Protocol=Self-refinement2026.0732—7,900
Competitive, shared opponent (single-pass)Model=Qwen3-0.6B, Inference Passes=single-pass, Training Protocol=Competitive2026.0732—7,400
Vanilla GRPOModel=Qwen3-0.6B, Protocol=single training run, cascade protocol2026.0730——
Vanilla GRPO (baseline)Model=Qwen3-0.6B, Inference Passes=single-pass, Training Protocol=GRPO2026.0730—8,100
Zero-shotModel=Qwen3-0.6B, Protocol=single evaluation2026.0723——
Zero-shotModel=Qwen3-0.6B, Inference Passes=single-pass, Training=None2026.0723—6,100