List your company
ResearchFormerly WizWand

Mathematical Reasoning on AIME 24 (ACC, LEN)

79.17AccuracyCOPT
-3.166818.209139.58560.9609Jun 9, 2025Aug 9, 2025Oct 9, 2025Dec 9, 2025Feb 8, 2026Apr 10, 2026Jun 11, 2026
Updated 4mo ago

Evaluation results

MethodLinks
COPTBackbone=Qwen3-8B, Reasoning Effort=High (Peak Accuracy)2026.0579.1711,525—
CoTBackbone=Qwen3-8B2026.0575.8312,077—
CoT (Greedy)Backbone=Qwen3-8B2026.057011,680—
SWA8k-RL-1200Training Stage=RL, RL Steps=1200, RL Training Budget=~500 GPU Hours, Attention Architecture=Sliding Window Attention, Sliding Window Size=8k, T_Train=466, T_Eval=2.232026.0660.3——
SWA4k-RL-1400Training Stage=RL, RL Steps=1400, RL Training Budget=~500 GPU Hours, Attention Architecture=Sliding Window Attention, Sliding Window Size=4k, T_Train=474, T_Eval=1.872026.0657.5——
SA-RL-900Training Stage=RL, RL Steps=900, Attention Architecture=Standard Attention, T_Train=498, T_Eval=2.812026.0656.6——
SWA8k-RL-900Training Stage=RL, RL Steps=900, Attention Architecture=Sliding Window Attention, Sliding Window Size=8k, T_Train=337, T_Eval=2.112026.0656.6——
SWA4k-RL-900Training Stage=RL, RL Steps=900, Attention Architecture=Sliding Window Attention, Sliding Window Size=4k, T_Train=285, T_Eval=1.842026.0653——
SWA2k-RL-1700Training Stage=RL, RL Steps=1700, RL Training Budget=~500 GPU Hours, Attention Architecture=Sliding Window Attention, Sliding Window Size=2k, T_Train=470, T_Eval=1.332026.0652.4——
SWA2k-RL-900Training Stage=RL, RL Steps=900, Attention Architecture=Sliding Window Attention, Sliding Window Size=2k, T_Train=225, T_Eval=1.462026.0646.8——
Qwen3-14B + GRPOBackbone=Qwen3-14B, Training=GRPO2026.0546.5—6,362
Qwen3-14B + SLATBackbone=Qwen3-14B, Training=SLAT2026.0545—4,198
SFPOModel=DS-Qwen-7B2025.1043.75——
GRPOModel=DS-Qwen-7B2025.1042.5——
SA-SFTTraining Stage=SFT, Attention Architecture=Standard Attention, T_Train=615, T_Eval=10.442026.0640.1——
SWA8k-SFTTraining Stage=SFT, Attention Architecture=Sliding Window Attention, Sliding Window Size=8k, T_Train=560, T_Eval=5.542026.0634.2——
SFPOModel=DS-Qwen-1.5B2025.1034.17——
GRPOModel=DS-Qwen-1.5B2025.1032.92——
Qwen3-14BBackbone=Qwen3-14B2026.0528.8—1,641
LRSModel=OPEN-REASONER-7B, Prompting=0-shot2026.0526.6——
BaseModel=DS-Qwen-7B2025.1025.42——
ReSumBackbone=Qwen2.5-Math-7B2026.0625.42——
DGPOBackbone=Qwen2.5-Math-7B2026.0623.85——
SWA4k-SFTTraining Stage=SFT, Attention Architecture=Sliding Window Attention, Sliding Window Size=4k, T_Train=528, T_Eval=4.702026.0622.4——
GPGBackbone=Qwen2.5-Math-7B2026.0621.98——
GRPO-ADBackbone=Qwen2.5-Math-7B2026.0621.56——
DAPOBackbone=Qwen2.5-Math-7B2026.0621.25——
Dr. GRPOBackbone=Qwen2.5-Math-7B2026.0621.04——
GRPOBackbone=Qwen2.5-Math-7B2026.0620.94——
BaseModel=DS-Qwen-1.5B2025.1020.4——
GSPOBackbone=Qwen2.5-Math-7B2026.0619.38——
ReLIFTBackbone=Qwen2.5-7B2025.0619.15,522—
STEERBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@322025.1017.4——
BaseModel=OPEN-REASONER-7B, Prompting=0-shot2026.0516.6——
LRS BASICModel=OPEN-REASONER-7B, Prompting=0-shot2026.0516.6——
GRPOBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@322025.1016.2——
ReSumBackbone=Qwen2.5-Math-1.5B2026.0615.96——
SFTBackbone=Qwen2.5-7B2025.0615.77,786—
RLBackbone=Qwen2.5-7B2025.0615.51,784—
Entro. Adv.Backbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@322025.1015——
OPOBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@322025.1014.8——
Clip-CovBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@322025.1014.7——
ReLIFTBackbone=Qwen2.5-Math-1.5B2025.0614.33,691—
SWA2k-SFTTraining Stage=SFT, Attention Architecture=Sliding Window Attention, Sliding Window Size=2k, T_Train=507, T_Eval=4.202026.0613.8——
CoTModel=OPEN-REASONER-7B2026.0513.3——
few-shotModel=OPEN-REASONER-7B2026.0513.3——
LRSModel=OPEN-REASONER-1.5B, Prompting=0-shot2026.0513.3——
SFTBackbone=Qwen2.5-Math-1.5B2025.0612.77,925—
Base ModelBackbone=Qwen2.5-Math-7B2026.0612.19——
Qwen-7B-InstructBackbone=Qwen2.5-7B2025.0611.51,939—
GRPOBackbone=Qwen2.5-Math-1.5B2026.0611.35——
DGPOBackbone=Qwen2.5-Math-1.5B2026.0611.25——
Qwen-Math-1.5B-InstructBackbone=Qwen2.5-Math-1.5B2025.0610.34,042—
BaseBase Model=LLaMA 3.1-8B-Instruct, Optimization Setting=Training-time optimization, Reward Supervision=Noisy rule-based2025.0810——
VRPOBase Model=LLaMA 3.1-8B-Instruct, Optimization Setting=Training-time optimization, Reward Supervision=Noisy rule-based2025.0810——
RLBackbone=Qwen2.5-Math-1.5B2025.069.82,425—
ReSumBackbone=Qwen2.5-3B2026.067.5——
DGPOBackbone=Qwen2.5-3B2026.066.98——
Base ModelBackbone=Qwen2.5-Math-1.5B2026.066.87——
CoTModel=OPEN-REASONER-1.5B2026.056.7——
few-shotModel=OPEN-REASONER-1.5B2026.056.7——
PPOBase Model=LLaMA 3.1-8B-Instruct, Optimization Setting=Training-time optimization, Reward Supervision=Noisy rule-based2025.086.67——
Qwen-7BBackbone=Qwen2.5-7B2025.066.21,632—
LLaMa-8B-InstructBackbone=LLaMA-3.1-8B2025.065.61,396—
GRPOBackbone=Qwen2.5-3B2026.065.31——
BaseBackbone=Qwen2.5-Math-1.5B, Sampling Strategy=avg@322025.104.1——
GRPOBase Model=LLaMA 3.1-8B-Instruct, Optimization Setting=Training-time optimization, Reward Supervision=Noisy rule-based2025.083.33——
BaseModel=OPEN-REASONER-1.5B, Prompting=0-shot2026.053.3——
Base ModelBackbone=Qwen2.5-3B2026.062.81——
Qwen-Math-1.5BBackbone=Qwen2.5-Math-1.5B2025.062.81,971—
DGPOBackbone=DeepSeek-Math-7B2026.061.98——
RLBackbone=LLaMA-3.1-8B2025.061.8940—
ReSumBackbone=DeepSeek-Math-7B2026.061.66——
ReLIFTBackbone=LLaMA-3.1-8B2025.061.31,236—
SFTBackbone=LLaMA-3.1-8B2025.060.82,033—
GRPOBackbone=DeepSeek-Math-7B2026.060.63——
Base ModelBackbone=DeepSeek-Math-7B2026.060.42——
LRS BASICModel=OPEN-REASONER-1.5B, Prompting=0-shot2026.050——