List your company
ResearchFormerly WizWand

Mathematical Reasoning on MATH-500 (Accuracy, Token Length)

97.6AccuracyGRPO
27.08845.39463.782.006Jul 20, 2025Sep 10, 2025Nov 1, 2025Dec 23, 2025Feb 13, 2026Apr 6, 2026May 28, 2026
Updated 5mo ago

Evaluation results

MethodLinks
GRPOBackbone=Qwen3-4B2026.0497.64,974
VanillaBackbone=Qwen3-8B2026.0496.85,299
GRPOBackbone=Qwen3-8B2026.0496.85,092
Step-GRPOBackbone=Qwen3-8B2026.0496.83,485
MC²Backbone=Gemini-2.0-flash, Reasoning Method=MC²2026.0496.731,299
GRPO-λBackbone=Qwen3-4B2026.0496.63,232
GRPO-λBackbone=Qwen3-8B2026.0496.43,095
GRPO-8kBackbone=Qwen3-8B2026.0496.23,796
VanillaBackbone=Qwen3-4B2026.0496.25,346
GRPO-8kBackbone=Qwen3-4B2026.0496.23,691
GRPO+SOPBackbone=Qwen3-8B2026.0495.83,822
GRPO+SOPBackbone=Qwen3-4B2026.0495.63,808
Step-GRPOBackbone=Qwen3-4B2026.0495.63,528
DEER+SFTBackbone=Qwen3-8B2026.04953,361
GRPO+LPBackbone=Qwen3-8B2026.04952,336
ToTBackbone=Gemini-2.0-flash, Reasoning Method=ToT2026.0494.931,999
Per-Step ScaleBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.0794.65,227
GRPO+LPBackbone=Qwen3-4B2026.0494.22,329
CoT-SCBackbone=Gemini-2.0-flash, Reasoning Method=CoT-SC2026.0494.23,018
SMARTBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07944,932
MURBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.07943,607
Avg uncertaintyBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.0793.83,846
MURBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.0793.85,328
MC²Backbone=Qwen3-8B, Reasoning Method=MC²2026.0493.071,582
DEER+SFTBackbone=Qwen3-4B2026.04933,610
SMARTBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.07935,482
GRPOBackbone=Qwen3-1.7B2026.0492.45,140
Buffer of ThoughtBackbone=Gemini-2.0-flash, Reasoning Method=Buffer of Thought2026.0492.272,192
Meta PromptBackbone=Qwen3-8B, Reasoning Method=Meta Prompt2026.0491.931,545
Meta ReasoningBackbone=Gemini-2.0-flash, Reasoning Method=Meta Reasoning2026.0491.73638
CoTBackbone=Gemini-2.0-flash, Reasoning Method=CoT2026.0491.53602
Meta PromptBackbone=Gemini-2.0-flash, Reasoning Method=Meta Prompt2026.0491.071,465
VanillaBackbone=Qwen3-1.7B2026.0490.85,777
GRPO-8kBackbone=Qwen3-1.7B2026.0490.63,931
Avg uncertaintyBackbone=Qwen3-8B, Reasoning Mode=Thinking Mode2025.0790.64,385
GRPO+SOPBackbone=Qwen3-1.7B2026.0490.24,419
GRPO-λBackbone=Qwen3-1.7B2026.0490.23,708
Step-GRPOBackbone=Qwen3-1.7B2026.0490.23,812
SMARTBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.0789.65,214
GRPO+LPBackbone=Qwen3-1.7B2026.0489.42,718
MURBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.0789.25,041
Per-Step ScaleBackbone=Qwen3-4B, Reasoning Mode=Thinking Mode2025.0789.24,598
Avg uncertaintyBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.0788.84,528
CoT-SCBackbone=Qwen3-8B, Reasoning Method=CoT-SC2026.0487.934,199
Per-Step ScaleBackbone=Qwen3-1.7B, Reasoning Mode=Thinking Mode2025.0787.65,841
ToTBackbone=Qwen3-8B, Reasoning Method=ToT2026.0487.533,232
Buffer of ThoughtBackbone=Qwen3-8B, Reasoning Method=Buffer of Thought2026.0486.61,983
Meta ReasoningBackbone=Qwen3-8B, Reasoning Method=Meta Reasoning2026.0485.27978
MC²Backbone=GPT-4o-mini, Reasoning Method=MC²2026.0485.131,631
CoTBackbone=Qwen3-8B, Reasoning Method=CoT2026.0484.4932
TTPIBackbone=Qwen3-8B, Reasoning Method=TTPI2026.0483.73888
DEER+SFTBackbone=Qwen3-1.7B2026.04834,615
Non-Thinking ModeBackbone=Qwen3-8B, Reasoning Mode=Non-Thinking Mode2025.0781.41,131
Non-Thinking ModeBackbone=Qwen3-4B, Reasoning Mode=Non-Thinking Mode2025.0779.4772
CoT-SCBackbone=GPT-4o-mini, Reasoning Method=CoT-SC2026.0478.933,200
ToTBackbone=GPT-4o-mini, Reasoning Method=ToT2026.0477.63,473
MC²Backbone=Llama-3.1-8B-Instruct, Reasoning Method=MC²2026.0476.84,435
Meta ReasoningBackbone=GPT-4o-mini, Reasoning Method=Meta Reasoning2026.0476.4657
NumcaBase Model=Qwen2.5-Math-1.5B-Instruct, RL Algorithm=GRPO, Credit Assignment Method=Numca2026.0576—
Meta PromptBackbone=GPT-4o-mini, Reasoning Method=Meta Prompt2026.0475.731,727
Buffer of ThoughtBackbone=GPT-4o-mini, Reasoning Method=Buffer of Thought2026.0475.331,756
GRPOBase Model=Qwen2.5-Math-1.5B-Instruct, RL Algorithm=GRPO2026.0574.6—
TTPIBackbone=GPT-4o-mini, Reasoning Method=TTPI2026.0474.331,208
Qwen2.5-Math-1.5B-InstructBase Model=Qwen2.5-Math-1.5B-Instruct2026.0574—
CoTBackbone=GPT-4o-mini, Reasoning Method=CoT2026.0472.67625
Vanilla CoTBackbone=Qwen3-1.7B, Reasoning Mode=Non-Thinking Mode2025.0769.21,047
Meta PromptBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Prompt2026.0465.474,426
CoT-SCBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT-SC2026.0461.477,652
ToTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=ToT2026.0460.476,450
CoTBackbone=Llama-3.1-8B-Instruct, Reasoning Method=CoT2026.0456.271,942
Buffer of ThoughtBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Buffer of Thought2026.0455.85,564
TTPIBackbone=Llama-3.1-8B-Instruct, Reasoning Method=TTPI2026.0454.84,284
Meta ReasoningBackbone=Llama-3.1-8B-Instruct, Reasoning Method=Meta Reasoning2026.0454.672,117
SDBackbone=Llama-3.1-8B/Llama-3.2-1B2025.0945.49.98
SelfJudge-RBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best recall2025.094310.91
SelfJudge-FBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best F1 score2025.0942.212.78
AutoJudge-RBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best recall2025.094111.23
AutoJudge-FBackbone=Llama-3.1-8B/Llama-3.2-1B, Thresholding strategy=best F1 score2025.0940.613.12
Top-kBackbone=Llama-3.1-8B/Llama-3.2-1B, k=22025.0929.815.08