List your company
ResearchFormerly WizWand

Mathematical Reasoning on AIME 25 (Acc, Inference Cost, Cost Reduction)

94.7AccuracyThink@n
-0.324824.345149.01573.6849Sep 27, 2025Oct 26, 2025Nov 25, 2025Dec 24, 2025Jan 23, 2026Feb 21, 2026Mar 23, 2026
Updated 7mo ago

Evaluation results

MethodLinks
Think@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.0294.7155.4-49
Cons@nBackbone=OSS-120B-medium, Aggregation Method=Standard self-consistency2026.0292.7307.6—
Short@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over shortest responses2026.0290983.6-8
Think@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.0290537.5-50
Short@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over shortest responses2026.0287.3255.7-17
Self-Certainty@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.0287.3150.6-51
Long@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over longest responses2026.0286.7307.6—
Cons@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Standard self-consistency2026.0286.71,073.1—
Self-Certainty@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.0286.7548.9-49
Long@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over longest responses2026.0285.31,073.1—
Mean@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Average accuracy2026.0281.21,073.1—
Mean@nBackbone=OSS-120B-medium, Aggregation Method=Average accuracy2026.0280307.6—
GOLFBackbone=Qwen-3-8B2026.0341.65——
GOLFBackbone=Qwen-3-4B2026.0338.1——
GRPOBackbone=Qwen-3-8B2026.0338.02——
Critique-GRPOBackbone=Qwen-3-8B2026.0337.86——
ROSA (+LM + M)Model=Qwen3-8B2025.0936.67——
Critique-GRPOBackbone=Qwen-3-4B2026.0335.89——
GRPOBackbone=Qwen-3-4B2026.0335.42——
ROSA (+HS + R)Model=Qwen3-8B2025.0933.33——
ROSA (+LM + R)Model=Qwen3-8B2025.0930——
Critique-FTBackbone=Qwen-3-8B2026.0328.75——
Refinement-FTBackbone=Qwen-3-8B2026.0327.5——
Critique-FTBackbone=Qwen-3-4B2026.0324.58——
ROSA (+LM + R)Model=Qwen2.5-7B-Instruct2025.0923.33——
Refinement-FTBackbone=Qwen-3-4B2026.0321.25——
ROSA (+HS + R)Model=Qwen2.5-7B-Instruct2025.0920——
ROSA (+LM + M)Model=Qwen2.5-7B-Instruct2025.0920——
Qwen-3-8BTraining Method=SFT2026.0319.6——
Qwen-3-4BTraining Method=SFT2026.0318.55——
ROSA (+LM + R)Model=Qwen3-0.6B2025.0916.67——
BaselineModel=Qwen3-8B2025.0916.67——
ROSA (+LM + R)Model=DeepSeek-R1-Distill-Llama-8B2025.0916.67——
ROSA (+HS + R)Model=DeepSeek-R1-Distill-Llama-8B2025.0916.67——
ROSA (+LM + M)Model=DeepSeek-R1-Distill-Llama-8B2025.0916.67——
ReMAModel=qwen3-4b-base, Agent Configuration=Dual-Agent2026.0313.33——
ReMAModel=qwen2.5-7b-instruct, Agent Configuration=Dual-Agent2026.0313.3——
BaselineModel=Qwen3-0.6B2025.0910——
ROSA (+HS + R)Model=Qwen3-0.6B2025.0910——
ROSA (+LM + M)Model=Qwen3-0.6B2025.0910——
BaselineModel=Qwen2.5-7B-Instruct2025.0910——
UntrainedModel=qwen2.5-7b-instruct, Agent Configuration=Single-agent2026.0310——
CCPOModel=qwen2.5-7b-instruct, Agent Configuration=Dual-Agent2026.0310——
ROSA (+LM + R)Model=Qwen2.5-0.5B-Instruct2025.096.67——
ROSA (+HS + R)Model=Qwen2.5-0.5B-Instruct2025.096.67——
ROSA (+LM + M)Model=Qwen2.5-0.5B-Instruct2025.096.67——
UntrainedModel=qwen2.5-1.5b-instruct, Agent Configuration=Dual-Agent2026.036.67——
GRPOModel=qwen2.5-7b-instruct, Agent Configuration=Single-agent2026.036.67——
UntrainedModel=qwen2.5-7b-instruct, Agent Configuration=Dual-Agent2026.036.67——
GRPOModel=qwen3-4b-base, Agent Configuration=Single-agent2026.036.67——
UntrainedModel=qwen3-4b-base, Agent Configuration=Dual-Agent2026.036.67——
CCPOModel=qwen3-4b-base, Agent Configuration=Dual-Agent2026.036.67——
BaselineModel=Qwen2.5-0.5B-Instruct2025.093.33——
BaselineModel=DeepSeek-R1-Distill-Llama-8B2025.093.33——