List your company
ResearchFormerly WizWand

Mathematical Reasoning on AIME 24 (Accuracy, Inference Cost, Cost Reduction)

93.3AccuracyThink@n
-2.79622.15247.172.048Feb 10, 2026Feb 11, 2026Feb 12, 2026Feb 13, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Think@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.0293.3121.3-48
Cons@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Standard self-consistency2026.0293.3950.1—
Think@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.0293.3482.2-49
Cons@nBackbone=OSS-120B-medium, Aggregation Method=Standard self-consistency2026.0292.7235.1—
Self-Certainty@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.0291.3119.3-49
Short@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over shortest responses2026.0290871-8
Self-Certainty@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.0290480.9-49
Short@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over shortest responses2026.0288200.9-15
Long@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over longest responses2026.0286.7235.1—
Long@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over longest responses2026.0286.7950.1—
Mean@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Average accuracy2026.0286.3950.1—
Mean@nBackbone=OSS-120B-medium, Aggregation Method=Average accuracy2026.0281.6235.1—
LaPha / sg@128Base model=Qwen2.5-Math-7B, Tool=✓2026.0260——
LaPha / sg@128Base model=Qwen2.5-Math-1.5B, Tool=✓2026.0256.7——
GPT-o1-miniTool=✗2026.0256.7——
LaPha / sg@128Base model=Qwen2.5-7B, Tool=✓2026.0246.7——
ToRLBase model=Qwen2.5-Math-7B, Tool=✓2026.0243.3——
DAPOBase model=Qwen2.5-Math-7B, Tool=✗2026.0236.7——
SimpleRLBase model=Qwen2.5-Math-7B, Tool=✗2026.0233.3——
LaPhaBase model=Qwen2.5-Math-1.5B, Tool=✓2026.0230——
TreePO / maj@16Base model=Qwen2.5-7B, Tool=✗2026.0228.9——
ToRLBase model=Qwen2.5-Math-1.5B, Tool=✓2026.0226.7——
SFTBase model=Qwen2.5-Math-7B, Tool=✓2026.0226.7——
PrimeBase model=Qwen2.5-Math-7B, Tool=✗2026.0226.7——
SFTBase model=Qwen2.5-Math-1.5B, Tool=✓2026.0223.3——
LaPha / sg@128Base model=Qwen2.5-1.5B, Tool=✓2026.0220——
DAPOBase model=Qwen2.5-Math-1.5B, Tool=✗2026.0220——
DAPOBase model=Qwen2.5-7B, Tool=✗2026.0216.7——
LaPhaBase model=Qwen2.5-1.5B, Tool=✓2026.0212.7——
SFTBase model=Qwen2.5-Math-7B, Tool=✗2026.0210——
GPT-4oTool=✗2026.029.3——
SFTBase model=Qwen2.5-7B, Tool=✗2026.029.1——
DAPOBase model=Qwen2.5-1.5B, Tool=✗2026.026.7——
SFTBase model=Qwen2.5-Math-1.5B, Tool=✗2026.023.3——
SFTBase model=Qwen2.5-1.5B, Tool=✗2026.020.9——