Mathematical Reasoning on MATH-500 (Accuracy, Token Length)
97.6AccuracyGRPO
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GRPOBackbone=Qwen3-4B2026.04 | 97.6 | 4,974 | |
| VanillaBackbone=Qwen3-8B2026.04 | 96.8 | 5,299 | |
| GRPOBackbone=Qwen3-8B2026.04 | 96.8 | 5,092 | |
| Step-GRPOBackbone=Qwen3-8B2026.04 | 96.8 | 3,485 | |
| MC²Backbone=Gemini-2.0-flash, Reasoning Method=MC²2026.04 | 96.73 | 1,299 | |
| GRPO-λBackbone=Qwen3-4B2026.04 | 96.6 | 3,232 | |
| GRPO-λBackbone=Qwen3-8B2026.04 | 96.4 | 3,095 | |
| GRPO-8kBackbone=Qwen3-8B2026.04 | 96.2 | 3,796 | |
| VanillaBackbone=Qwen3-4B2026.04 | 96.2 | 5,346 | |
| GRPO-8kBackbone=Qwen3-4B2026.04 | 96.2 | 3,691 | |
| GRPO+SOPBackbone=Qwen3-8B2026.04 | 95.8 | 3,822 | |
| GRPO+SOPBackbone=Qwen3-4B2026.04 | 95.6 | 3,808 | |
| Step-GRPOBackbone=Qwen3-4B2026.04 | 95.6 | 3,528 | |
| DEER+SFTBackbone=Qwen3-8B2026.04 | 95 | 3,361 | |
| GRPO+LPBackbone=Qwen3-8B2026.04 | 95 | 2,336 | |
| ToTBackbone=Gemini-2.0-flash, Reasoning Method=ToT2026.04 | 94.93 | 1,999 | |
| Per-Step ScaleBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | 94.6 | 5,227 | |
| GRPO+LPBackbone=Qwen3-4B2026.04 | 94.2 | 2,329 | |
| CoT-SCBackbone=Gemini-2.0-flash, Reasoning Method=CoT-SC2026.04 | 94.2 | 3,018 | |
| SMARTBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | 94 | 4,932 | |
| MURBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | 94 | 3,607 | |
| Avg uncertaintyBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | 93.8 | 3,846 | |
| MURBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | 93.8 | 5,328 | |
| MC²Backbone=Qwen3-8B, Reasoning Method=MC²2026.04 | 93.07 | 1,582 | |
| DEER+SFTBackbone=Qwen3-4B2026.04 | 93 | 3,610 | |
| SMARTBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | 93 | 5,482 | |
| GRPOBackbone=Qwen3-1.7B2026.04 | 92.4 | 5,140 | |
| Buffer of ThoughtBackbone=Gemini-2.0-flash, Reasoning Method=Buffer of Thought2026.04 | 92.27 | 2,192 | |
| Meta PromptBackbone=Qwen3-8B, Reasoning Method=Meta Prompt2026.04 | 91.93 | 1,545 | |
| Meta ReasoningBackbone=Gemini-2.0-flash, Reasoning Method=Meta Reasoning2026.04 | 91.73 | 638 | |
| CoTBackbone=Gemini-2.0-flash, Reasoning Method=CoT2026.04 | 91.53 | 602 | |
| Meta PromptBackbone=Gemini-2.0-flash, Reasoning Method=Meta Prompt2026.04 | 91.07 | 1,465 | |
| VanillaBackbone=Qwen3-1.7B2026.04 | 90.8 | 5,777 | |
| GRPO-8kBackbone=Qwen3-1.7B2026.04 | 90.6 | 3,931 | |
| Avg uncertaintyBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07 | 90.6 | 4,385 | |
| GRPO+SOPBackbone=Qwen3-1.7B2026.04 | 90.2 | 4,419 | |
| GRPO-λBackbone=Qwen3-1.7B2026.04 | 90.2 | 3,708 | |
| Step-GRPOBackbone=Qwen3-1.7B2026.04 | 90.2 | 3,812 | |
| SMARTBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | 89.6 | 5,214 | |
| GRPO+LPBackbone=Qwen3-1.7B2026.04 | 89.4 | 2,718 | |
| MURBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | 89.2 | 5,041 | |
| Per-Step ScaleBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07 | 89.2 | 4,598 | |
| Avg uncertaintyBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | 88.8 | 4,528 | |
| CoT-SCBackbone=Qwen3-8B, Reasoning Method=CoT-SC2026.04 | 87.93 | 4,199 | |
| Per-Step ScaleBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.07 | 87.6 | 5,841 | |
| ToTBackbone=Qwen3-8B, Reasoning Method=ToT2026.04 | 87.53 | 3,232 | |
| Buffer of ThoughtBackbone=Qwen3-8B, Reasoning Method=Buffer of Thought2026.04 | 86.6 | 1,983 | |
| Meta ReasoningBackbone=Qwen3-8B, Reasoning Method=Meta Reasoning2026.04 | 85.27 | 978 | |
| MC²Backbone=GPT-4o-mini, Reasoning Method=MC²2026.04 | 85.13 | 1,631 | |
| CoTBackbone=Qwen3-8B, Reasoning Method=CoT2026.04 | 84.4 | 932 | |
| TTPIBackbone=Qwen3-8B, Reasoning Method=TTPI2026.04 | 83.73 | 888 | |
| DEER+SFTBackbone=Qwen3-1.7B2026.04 | 83 | 4,615 | |
| Non-Thinking ModeBackbone=Qwen3-8B, Reasoning Mode=Non-Thinking Mode2025.07 | 81.4 | 1,131 | |
| Non-Thinking ModeBackbone=Qwen3-4B, Reasoning Mode=Non-Thinking Mode2025.07 | 79.4 | 772 | |
| CoT-SCBackbone=GPT-4o-mini, Reasoning Method=CoT-SC2026.04 | 78.93 | 3,200 | |
| ToTBackbone=GPT-4o-mini, Reasoning Method=ToT2026.04 | 77.6 | 3,473 | |
| MC²Backbone=Llama-3.1-8B-Instruct, Reasoning Method=MC²2026.04 | 76.8 | 4,435 | |
| Meta ReasoningBackbone=GPT-4o-mini, Reasoning Method=Meta Reasoning2026.04 | 76.4 | 657 | |
| NumcaBase Model=Qwen2.5-Math-1.5B-Instruct, RL Algorithm=GRPO, Credit Assignment Method=Numca2026.05 | 76 | — | |
| Meta PromptBackbone=GPT-4o-mini, Reasoning Method=Meta Prompt2026.04 | 75.73 | 1,727 | |
| Buffer of ThoughtBackbone=GPT-4o-mini, Reasoning Method=Buffer of Thought2026.04 | 75.33 | 1,756 | |
| GRPOBase Model=Qwen2.5-Math-1.5B-Instruct, RL Algorithm=GRPO2026.05 | 74.6 | — | |
| TTPIBackbone=GPT-4o-mini, Reasoning Method=TTPI2026.04 | 74.33 | 1,208 | |
| Qwen2.5-Math-1.5B-InstructBase Model=Qwen2.5-Math-1.5B-Instruct2026.05 | 74 | — | |
| CoTBackbone=GPT-4o-mini, Reasoning Method=CoT2026.04 | 72.67 | 625 | |
| Vanilla CoTBackbone=Qwen3-1.7B, Reasoning Mode=Non-Thinking Mode2025.07 | 69.2 | 1,047 | |
| Meta PromptBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Prompt2026.04 | 65.47 | 4,426 | |
| CoT-SCBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT-SC2026.04 | 61.47 | 7,652 | |
| ToTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=ToT2026.04 | 60.47 | 6,450 | |
| CoTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT2026.04 | 56.27 | 1,942 | |
| Buffer of ThoughtBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Buffer of Thought2026.04 | 55.8 | 5,564 | |
| TTPIBackbone=Llama-3.1-8B-Instruct, Reasoning Method=TTPI2026.04 | 54.8 | 4,284 | |
| Meta ReasoningBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Reasoning2026.04 | 54.67 | 2,117 | |
| SDBackbone=Llama-3.1-8B/Llama-3.2-1B2025.09 | 45.4 | 9.98 | |
| SelfJudge-RBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best recall2025.09 | 43 | 10.91 | |
| SelfJudge-FBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best F1 score2025.09 | 42.2 | 12.78 | |
| AutoJudge-RBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best recall2025.09 | 41 | 11.23 | |
| AutoJudge-FBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best F1 score2025.09 | 40.6 | 13.12 | |
| Top-kBackbone=Llama-3.1-8B/Llama-3.2-1B, k=22025.09 | 29.8 | 15.08 |