List your company
ResearchFormerly WizWand

Mathematical Reasoning on Olympiad Bench (Pass@1)

62.1Pass@1EVOTD
10.72424.06237.450.738Sep 28, 2025Nov 4, 2025Dec 12, 2025Jan 19, 2026Feb 25, 2026Apr 4, 2026May 12, 2026
Updated 5mo ago

Evaluation results

MethodLinks
EVOTDEvaluation Protocol=Pass@8, Training Iteration=Iter32026.0562.1
EVOTDEvaluation Protocol=Pass@8, Training Iteration=Iter22026.0561.5
EVOTDEvaluation Protocol=Pass@8, Training Iteration=Iter12026.0559.7
Evol-InstructEvaluation Protocol=Pass@82026.0559.1
Qwen3-4B (Thinking)Evaluation Protocol=Pass@82026.0559
EVOTDEvaluation Protocol=Pass@1, Training Iteration=Iter32026.0549.4
EVOTDEvaluation Protocol=Pass@1, Training Iteration=Iter22026.0549.1
EVOTDEvaluation Protocol=Pass@1, Training Iteration=Iter12026.0547.3
Evol-InstructEvaluation Protocol=Pass@12026.0546.9
Qwen3-4B (Thinking)Evaluation Protocol=Pass@12026.0545.6
BaselineBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.1041.5
SFTBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.1040.7
LightRBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.1039
LightRBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.1037.8
BaselineBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.1037.5
SFTBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.1037.5
LightRBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.1036.5
Qwen2.5-7B + GRPOBase Model=Qwen2.5-7B, RL Framework=GRPO, VERL Integration=false2025.0936.1
Qwen2.5-7B + PPOBase Model=Qwen2.5-7B, RL Framework=PPO, VERL Integration=false2025.0936.1
Qwen2.5-7B + PPO w/ VERL.Base Model=Qwen2.5-7B, RL Framework=PPO, VERL Integration=true2025.0936.1
Qwen2.5-7B + GRPO w/ VERL.Base Model=Qwen2.5-7B, RL Framework=GRPO, VERL Integration=true2025.0935.4
SFTBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.1027.6
LightRBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.1027.1
Qwen2.5-7BBase Model=Qwen2.5-7B, RL Framework=None, VERL Integration=false2025.0925.8
BaselineBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.1023.7
SFTBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.1021.2
SFTBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.1020.5
BaselineBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.1019.1
Llama-3.2-3B-Instruct + PPOBase Model=Llama-3.2-3B-Instruct, RL Framework=PPO, VERL Integration=false2025.0917.8
Llama-3.2-3B-Instruct + GRPO w/ VERL.Base Model=Llama-3.2-3B-Instruct, RL Framework=GRPO, VERL Integration=true2025.0917.6
Llama-3.2-3B-Instruct + PPO w/ VERL.Base Model=Llama-3.2-3B-Instruct, RL Framework=PPO, VERL Integration=true2025.0917.3
LightRBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.1016.9
Llama-3.2-3B-Instruct + GRPOBase Model=Llama-3.2-3B-Instruct, RL Framework=GRPO, VERL Integration=false2025.0916.7
BaselineBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.1016
Llama-3.2-3B-InstructBase Model=Llama-3.2-3B-Instruct, RL Framework=None, VERL Integration=false2025.0912.7