List your company
ResearchFormerly WizWand

Mathematical Reasoning on Olympiad Bench

68.13Pass@1 AccuracyMulFeRL
13.602827.758941.91556.0711Oct 24, 2024Jan 12, 2025Apr 2, 2025Jun 22, 2025Sep 10, 2025Nov 29, 2025Feb 18, 2026
Updated 8mo ago

Evaluation results

MethodLinks
MulFeRLBackbone=Qwen3-4B-Inst, Training Strategy=Multi-turn Feedback-guided Reinforcement Learning2026.0168.13
Critique-GRPO (CoT Critique)Backbone=Qwen3-8B2025.0666.8
Critique-GRPO (Self-Critique)w/ External Supervision=true2025.0666.2
R1-GRPOw/ External Supervision=true2025.0665.6
Critique-GRPO (Self-Critique & Self-Evaluation)w/ External Supervision=false2025.0665.5
GPT-o1-miniDecoding=Greedy2025.0265.3
Critique-GRPOBackbone=Qwen3-4B-Inst, Training Strategy=Reinforcement Learning-based Finetuning2026.0162.4
Dr.GRPOBackbone=Qwen3-4B-Inst, Training Strategy=Reinforcement Learning-based Finetuning2026.0160.36
Pass@8 (Upper Bound)Decoding Strategy=Pass@82025.0559.4
GRPOBackbone=Qwen3-4B-Inst, Training Strategy=Reinforcement Learning-based Finetuning2026.0159.05
SFTBackbone=Qwen3-4B-Inst, Training Strategy=Supervised Learning-based Finetuning2026.0149.17
Critique-GRPOTraining Data Volume=4k, Critique Mode=CoT-Critique2025.0648.6
CITL-FTBackbone=Qwen3-4B-Inst, Training Strategy=Supervised Learning-based Finetuning2026.0148.55
RAFTBackbone=Qwen3-4B-Inst, Training Strategy=Supervised Learning-based Finetuning2026.0148.34
SCOPEDecoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0546.8
SFTw/ External Supervision=true2025.0646.4
Majority @8Decoding Strategy=Majority @82025.0546.2
SCOPE (Step Skipping)Decoding Strategy=Best-of-8 strategy, Ablation=Step Skipping2025.0545.8
SCOPE (w/o AST normalization)Decoding Strategy=Best-of-8 strategy, Ablation=w/o AST normalization2025.0545.6
SCOPE (w/o code translation)Decoding Strategy=Best-of-8 strategy, Ablation=w/o code translation2025.0545.5
SCOPE (Step Replacement)Decoding Strategy=Best-of-8 strategy, Ablation=Step Replacement2025.0545.3
Qwen3-4B-InstBackbone=Qwen3-4B-Inst, Training Strategy=Base Model2026.0145.13
Qwen2.5-Math-7B-S2R-ORLBackbone=Qwen2.5-Math-7B, Training Strategy=Outcome-level RL, Decoding=Greedy2025.0244.9
RLHFlow-PRM-Deepseek-8BDecoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0544.3
Qwen3-8B (w/ Think)2025.0644.1
Qwen3-8BBackbone=Qwen3-8B2025.0644.1
Qwen2.5-Math-7B-PRM800KDecoding Strategy=Best-of-8 strategy, Training Data=manually annotated (PRM800K)2025.0543.5
Oat-ZeroTraining Data Volume=46k2025.0643.4
GPT-4oDecoding=Greedy2025.0243.3
RLHFlow-PRM-Mistral-8BDecoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0543.3
GPT-4o-2024-08-06Synthesis Model=-2024.1043.3
Skywork-PRM-1.5BDecoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0543
Skywork-PRM-7BDecoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0542.8
MulFeRLBackbone=Qwen2.5-7B-Base, Training Strategy=Multi-turn Feedback-guided Reinforcement Learning2026.0142.49
Critique-GRPO (CoT Critique)Backbone=Qwen2.5-7B-Base2025.0642.4
VI-CuRLBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Reward Supervision=Verifier-Free, Verifier Strategy=Majority Vote2026.0242
PRIME-ZeroTraining Data Volume=46k2025.0640.3
EurusPRM-Stage1Decoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0540.2
Progressive Thought EncodingBackbone Model=DeepSeek-R1-Distill-Llama-8B, Maximum TFLOPs of Attention=4.6, Peak GPU Mem. (%)=59.8, Mean GPU Mem. (%)=46.82026.0239.7
Math-Shepherd-PRM-7BDecoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0539.1
LoRABackbone Model=Qwen2.5-7B-Instruct, Maximum TFLOPs of Attention=5.7, Peak GPU Mem. (%)=85.8, Mean GPU Mem. (%)=59.32026.0238.7
Progressive Thought EncodingBackbone Model=Qwen2.5-7B-Instruct, Maximum TFLOPs of Attention=3.6, Peak GPU Mem. (%)=67.2, Mean GPU Mem. (%)=48.62026.0238.7
Critique-GRPOBackbone=Qwen2.5-7B-Base, Training Strategy=Reinforcement Learning-based Finetuning2026.0138.64
EurusPRM-Stage2Decoding Strategy=Best-of-8 strategy, Training Data=automatically constructed2025.0538.5
Qwen2-Math-7B-ScaleQuestBase Model=Qwen2-Math-7B, Synthesis Model=Qwen2-Math-7B-Instruct2024.1038.5
GreedyDecoding Strategy=Greedy2025.0538.2
Qwen2-Math-7B-InstructSynthesis Model=-2024.1037.8
GRPOBackbone=Qwen2.5-7B-Base, Training Strategy=Reinforcement Learning-based Finetuning2026.0136.77
MathScaleBackbone=Qwen2.5-7B-Instruct, Problem Crafter=ChatGPT, Chain-of-Thought=true, Zero-shot=true2025.0436.4
RV-SynBackbone=Qwen2.5-7B-Instruct, Problem Crafter=7B / 72B model, Chain-of-Thought=true, Zero-shot=true2025.0436.4
Official ModelBackbone=Qwen2.5-7B-Instruct, Problem Crafter=-, Chain-of-Thought=true, Zero-shot=true2025.0436
LoRAcBackbone Model=Qwen2.5-7B-Instruct, Maximum TFLOPs of Attention=3.5, Peak GPU Mem. (%)=63.1, Mean GPU Mem. (%)=45.42026.0235.9
Dr.GRPOBackbone=Qwen2.5-7B-Base, Training Strategy=Reinforcement Learning-based Finetuning2026.0135.79
Orca-MathBackbone=Qwen2.5-7B-Instruct, Problem Crafter=GPT-4, Chain-of-Thought=true, Zero-shot=true2025.0435.4
LoRABackbone Model=DeepSeek-R1-Distill-Llama-8B, Maximum TFLOPs of Attention=7.4, Peak GPU Mem. (%)=88.7, Mean GPU Mem. (%)=53.52026.0235.3
MetaMathBackbone=Qwen2.5-7B-Instruct, Problem Crafter=ChatGPT, Chain-of-Thought=true, Zero-shot=true2025.0435
VI-CuRLBackbone=Qwen2.5-Math-1.5B, Reward Supervision=Oracle Reward, Verifier Strategy=w. Verifier2026.0234.8
PromptCoTBackbone=Qwen2.5-7B-Instruct, Problem Crafter=72B Model, Chain-of-Thought=true, Zero-shot=true2025.0434.8
Mammoth2Backbone=Qwen2.5-7B-Instruct, Problem Crafter=72B Model, Chain-of-Thought=true, Zero-shot=true2025.0434.7
Jiuzhang3.0Backbone=Qwen2.5-7B-Instruct, Problem Crafter=7B Math Model, Chain-of-Thought=true, Zero-shot=true2025.0434.7
BaselineBackbone Model=Qwen2.5-7B-Instruct2026.0234.7
SimpleRL-ZeroTraining Data Volume=46k2025.0634.7
ScaleQuestBackbone=Qwen2.5-7B-Instruct, Problem Crafter=7B Math Model, Chain-of-Thought=true, Zero-shot=true2025.0434.4
Qwen2-Math-7B-Numina-MathBase Model=Qwen2-Math-7B, Synthesis Model=GPT-4o2024.1033.6
LoRAcBackbone Model=DeepSeek-R1-Distill-Llama-8B, Maximum TFLOPs of Attention=4.6, Peak GPU Mem. (%)=59.1, Mean GPU Mem. (%)=47.12026.0231.9
CITL-FTBackbone=Qwen2.5-7B-Base, Training Strategy=Supervised Learning-based Finetuning2026.0130.89
Qwen2.5-7B-BaseBackbone=Qwen2.5-7B-Base2025.0630.4
DeepSeekMath-7B-ScaleQuestBase Model=DeepSeekMath-7B, Synthesis Model=Qwen2-Math-7B-Instruct2024.1029.9
RAFTBackbone=Qwen2.5-7B-Base, Training Strategy=Supervised Learning-based Finetuning2026.0129.58
Progressive Thought EncodingBackbone Model=Qwen2.5-3B-Instruct, Maximum TFLOPs of Attention=2.7, Peak GPU Mem. (%)=45.3, Mean GPU Mem. (%)=32.62026.0229
BaselineBackbone Model=DeepSeek-R1-Distill-Llama-8B2026.0228.7
Qwen2.5-7B-BaseBackbone=Qwen2.5-7B-Base, Training Strategy=Base Model2026.0128.13
LoRABackbone Model=Qwen2.5-3B-Instruct, Maximum TFLOPs of Attention=4.2, Peak GPU Mem. (%)=82.8, Mean GPU Mem. (%)=63.52026.0227.8
LoRAcBackbone Model=Qwen2.5-3B-Instruct, Maximum TFLOPs of Attention=2.6, Peak GPU Mem. (%)=38, Mean GPU Mem. (%)=312026.0227.7
SFTBackbone=Qwen2.5-7B-Base, Training Strategy=Supervised Learning-based Finetuning2026.0127.27
BaselineBackbone Model=Qwen2.5-3B-Instruct2026.0227.2
Mistral-7B-ScaleQuestBase Model=Mistral-7B, Synthesis Model=Qwen2-Math-7B-Instruct2024.1026.8
Qwen2-7B-S2R-ORLBackbone=Qwen2-7B, Training Strategy=Outcome-level RL, Decoding=Greedy2025.0226.2
Llama3-8B-ScaleQuestBase Model=Llama3-8B, Synthesis Model=Qwen2-Math-7B-Instruct2024.1025.3
Qwen2-Math-7B-DART-MathBase Model=Qwen2-Math-7B, Synthesis Model=DSMath-7B-RL2024.1023.1
RV-SynBackbone=Phi-3-mini, Problem Crafter=7B / 72B model, Chain-of-Thought=true, Zero-shot=true2025.0422.4
ScaleQuestBackbone=Phi-3-mini, Problem Crafter=7B Math Model, Chain-of-Thought=true, Zero-shot=true2025.0421.8
DeepSeekMath-7B-DART-MathBase Model=DeepSeekMath-7B, Synthesis Model=DSMath-7B-RL2024.1021.7
Mammoth2Backbone=Phi-3-mini, Problem Crafter=72B Model, Chain-of-Thought=true, Zero-shot=true2025.0421.3
PromptCoTBackbone=Phi-3-mini, Problem Crafter=72B Model, Chain-of-Thought=true, Zero-shot=true2025.0421.3
Llama-3.1-8B-S2R-ORLBackbone=Llama-3.1-8B, Training Strategy=Outcome-level RL, Decoding=Greedy2025.0220.7
MetaMathBackbone=Phi-3-mini, Problem Crafter=ChatGPT, Chain-of-Thought=true, Zero-shot=true2025.0420.7
MathScaleBackbone=Phi-3-mini, Problem Crafter=ChatGPT, Chain-of-Thought=true, Zero-shot=true2025.0420.3
DeepSeekMath-7B-Numina-MathBase Model=DeepSeekMath-7B, Synthesis Model=GPT-4o2024.1019.9
Orca-MathBackbone=Phi-3-mini, Problem Crafter=GPT-4, Chain-of-Thought=true, Zero-shot=true2025.0419.6
Jiuzhang3.0Backbone=Phi-3-mini, Problem Crafter=7B Math Model, Chain-of-Thought=true, Zero-shot=true2025.0419.6
Mistral-7B-NuminaMathBase Model=Mistral-7B, Synthesis Model=GPT-4o2024.1019.4
DeepSeekMath-7B-RLSynthesis Model=-2024.1019
Qwen2-Math-7B-MetaMathBase Model=Qwen2-Math-7B, Synthesis Model=GPT-3.52024.1017.9
Llama3-8B-NuminaMathBase Model=Llama3-8B, Synthesis Model=GPT-4o2024.1017.8
Qwen2.5-Math-7B-BaseTraining Data Volume=None2025.0617.6
RV-SynBackbone=LLaMA-3-8B-Instruct, Problem Crafter=7B / 72B model, Chain-of-Thought=true, Zero-shot=true2025.0416.4
Jiuzhang3.0Backbone=LLaMA-3-8B-Instruct, Problem Crafter=7B Math Model, Chain-of-Thought=true, Zero-shot=true2025.0415.9
PromptCoTBackbone=LLaMA-3-8B-Instruct, Problem Crafter=72B Model, Chain-of-Thought=true, Zero-shot=true2025.0415.7
Official ModelBackbone=Phi-3-mini, Problem Crafter=-, Chain-of-Thought=true, Zero-shot=true2025.0415.7