Mathematical Reasoning on AIME 25 (Acc, Inference Cost, Cost Reduction)
94.7AccuracyThink@n
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Think@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.02 | 94.7 | 155.4 | -49 | |
| Cons@nBackbone=OSS-120B-medium, Aggregation Method=Standard self-consistency2026.02 | 92.7 | 307.6 | — | |
| Short@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over shortest responses2026.02 | 90 | 983.6 | -8 | |
| Think@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest DTR scores, Early Stopping Prefix Length=502026.02 | 90 | 537.5 | -50 | |
| Short@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over shortest responses2026.02 | 87.3 | 255.7 | -17 | |
| Self-Certainty@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.02 | 87.3 | 150.6 | -51 | |
| Long@nBackbone=OSS-120B-medium, Aggregation Method=Majority voting over longest responses2026.02 | 86.7 | 307.6 | — | |
| Cons@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Standard self-consistency2026.02 | 86.7 | 1,073.1 | — | |
| Self-Certainty@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over highest Self-Certainty score, Early Stopping Prefix Length=502026.02 | 86.7 | 548.9 | -49 | |
| Long@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Majority voting over longest responses2026.02 | 85.3 | 1,073.1 | — | |
| Mean@nBackbone=Qwen3-4B-Thinking, Aggregation Method=Average accuracy2026.02 | 81.2 | 1,073.1 | — | |
| Mean@nBackbone=OSS-120B-medium, Aggregation Method=Average accuracy2026.02 | 80 | 307.6 | — | |
| GOLFBackbone=Qwen-3-8B2026.03 | 41.65 | — | — | |
| GOLFBackbone=Qwen-3-4B2026.03 | 38.1 | — | — | |
| GRPOBackbone=Qwen-3-8B2026.03 | 38.02 | — | — | |
| Critique-GRPOBackbone=Qwen-3-8B2026.03 | 37.86 | — | — | |
| ROSA (+LM + M)Model=Qwen3-8B2025.09 | 36.67 | — | — | |
| Critique-GRPOBackbone=Qwen-3-4B2026.03 | 35.89 | — | — | |
| GRPOBackbone=Qwen-3-4B2026.03 | 35.42 | — | — | |
| ROSA (+HS + R)Model=Qwen3-8B2025.09 | 33.33 | — | — | |
| ROSA (+LM + R)Model=Qwen3-8B2025.09 | 30 | — | — | |
| Critique-FTBackbone=Qwen-3-8B2026.03 | 28.75 | — | — | |
| Refinement-FTBackbone=Qwen-3-8B2026.03 | 27.5 | — | — | |
| Critique-FTBackbone=Qwen-3-4B2026.03 | 24.58 | — | — | |
| ROSA (+LM + R)Model=Qwen2.5-7B-Instruct2025.09 | 23.33 | — | — | |
| Refinement-FTBackbone=Qwen-3-4B2026.03 | 21.25 | — | — | |
| ROSA (+HS + R)Model=Qwen2.5-7B-Instruct2025.09 | 20 | — | — | |
| ROSA (+LM + M)Model=Qwen2.5-7B-Instruct2025.09 | 20 | — | — | |
| Qwen-3-8BTraining Method=SFT2026.03 | 19.6 | — | — | |
| Qwen-3-4BTraining Method=SFT2026.03 | 18.55 | — | — | |
| ROSA (+LM + R)Model=Qwen3-0.6B2025.09 | 16.67 | — | — | |
| BaselineModel=Qwen3-8B2025.09 | 16.67 | — | — | |
| ROSA (+LM + R)Model=DeepSeek-R1-Distill-Llama-8B2025.09 | 16.67 | — | — | |
| ROSA (+HS + R)Model=DeepSeek-R1-Distill-Llama-8B2025.09 | 16.67 | — | — | |
| ROSA (+LM + M)Model=DeepSeek-R1-Distill-Llama-8B2025.09 | 16.67 | — | — | |
| ReMAModel=qwen3-4b-base, Agent Configuration=Dual-Agent2026.03 | 13.33 | — | — | |
| ReMAModel=qwen2.5-7b-instruct, Agent Configuration=Dual-Agent2026.03 | 13.3 | — | — | |
| BaselineModel=Qwen3-0.6B2025.09 | 10 | — | — | |
| ROSA (+HS + R)Model=Qwen3-0.6B2025.09 | 10 | — | — | |
| ROSA (+LM + M)Model=Qwen3-0.6B2025.09 | 10 | — | — | |
| BaselineModel=Qwen2.5-7B-Instruct2025.09 | 10 | — | — | |
| UntrainedModel=qwen2.5-7b-instruct, Agent Configuration=Single-agent2026.03 | 10 | — | — | |
| CCPOModel=qwen2.5-7b-instruct, Agent Configuration=Dual-Agent2026.03 | 10 | — | — | |
| ROSA (+LM + R)Model=Qwen2.5-0.5B-Instruct2025.09 | 6.67 | — | — | |
| ROSA (+HS + R)Model=Qwen2.5-0.5B-Instruct2025.09 | 6.67 | — | — | |
| ROSA (+LM + M)Model=Qwen2.5-0.5B-Instruct2025.09 | 6.67 | — | — | |
| UntrainedModel=qwen2.5-1.5b-instruct, Agent Configuration=Dual-Agent2026.03 | 6.67 | — | — | |
| GRPOModel=qwen2.5-7b-instruct, Agent Configuration=Single-agent2026.03 | 6.67 | — | — | |
| UntrainedModel=qwen2.5-7b-instruct, Agent Configuration=Dual-Agent2026.03 | 6.67 | — | — | |
| GRPOModel=qwen3-4b-base, Agent Configuration=Single-agent2026.03 | 6.67 | — | — | |
| UntrainedModel=qwen3-4b-base, Agent Configuration=Dual-Agent2026.03 | 6.67 | — | — | |
| CCPOModel=qwen3-4b-base, Agent Configuration=Dual-Agent2026.03 | 6.67 | — | — | |
| BaselineModel=Qwen2.5-0.5B-Instruct2025.09 | 3.33 | — | — | |
| BaselineModel=DeepSeek-R1-Distill-Llama-8B2025.09 | 3.33 | — | — |