List your company
ResearchFormerly WizWand

Mathematical Reasoning on Olympiad (Pass@1 accuracy)

68.3Pass@1 AccuracyPieceHint-Nemotron-1.5B
0.07617.78835.553.212Sep 29, 2025Nov 2, 2025Dec 7, 2025Jan 10, 2026Feb 14, 2026Mar 20, 2026Apr 24, 2026
Updated 5mo ago

Evaluation results

MethodLinks
PieceHint-Nemotron-1.5BModel Name=Nemotron-1.5B, Model Size=1.5B, Method Type=PieceHint2026.0468.3
SFPOModel=DS-distilled-Qwen-7B2025.1065.73
DeepSearch-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0965.72
Nemotron-Research-Reasoning-Qwen-1.5B v2number of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0964.69
Nemotron-Research-Reasoning-Qwen-1.5B v1number of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0964.56
PieceHint-Qwen3-1.7BModel Name=Qwen3-1.7B, Model Size=1.7B, Method Type=PieceHint2026.0462.4
GRPOModel=DS-distilled-Qwen-7B2025.1061.24
Nemotron-1.5BModel Name=Nemotron-1.5B, Model Size=1.5B, Method Type=Baseline2026.0461.2
DeepSeek-R1-Distill-32BModel Name=DeepSeek-R1-Distill-32B, Model Size=32B, Method Type=Baseline2026.0461.2
DeepScaleR-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0958.95
Qwen3-4BModel Name=Qwen3-4B, Model Size=4B, Method Type=Baseline2026.0458.5
DeepSeek-R1-Distill-7BModel Name=DeepSeek-R1-Distill-7B, Model Size=7B, Method Type=Baseline2026.0455.1
STILL-3-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0953.84
Qwen3-1.7BModel Name=Qwen3-1.7B, Model Size=1.7B, Method Type=Baseline2026.0453.7
Open-RS1-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0952.82
Open-RS2-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0952.63
Open-RS3-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0952.25
Qwen2.5-32B + SimpleRLEvaluation Approach=LLM-as-a-judge (Ours), Model Size=32B, Training Strategy=SimpleRL2026.0452
DeepSeek-R1-Distill-Qwen-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0951.55
SFPOModel=DS-distilled-Qwen-1.5B2025.1050.67
PieceHint-DeepSeek-1.5BModel Name=DeepSeek-R1-Distill-1.5B, Model Size=1.5B, Method Type=PieceHint2026.0450.1
GRPOModel=DS-distilled-Qwen-1.5B2025.1049.85
BaseModel=DS-distilled-Qwen-7B2025.1048.89
SFPOModel=Qwen3-4B-Base2025.1048.67
GRPOModel=Qwen3-4B-Base2025.1048.59
Qwen2.5-14B + SimpleRLEvaluation Approach=LLM-as-a-judge (Ours), Model Size=14B, Training Strategy=SimpleRL2026.0448.1
Qwen2.5-7B + SimpleRL (Lighteval)Evaluation Approach=LLM-as-a-judge (Ours), Model Size=7B, Training Strategy=SimpleRL, Evaluation Framework=Lighteval2026.0446.1
Qwen2.5-32B + SimpleRLEvaluation Approach=Symbolic (Baseline), Model Size=32B, Training Strategy=SimpleRL2026.0446
SFPOModel=Qwen2.5-Math-7B2025.1045.59
GRPOModel=Qwen2.5-Math-7B2025.1044.8
Qwen2.5-14B + SimpleRLEvaluation Approach=Symbolic (Baseline), Model Size=14B, Training Strategy=SimpleRL2026.0443.5
DeepSeek-R1-Distill-1.5BModel Name=DeepSeek-R1-Distill-1.5B, Model Size=1.5B, Method Type=Baseline2026.0443.4
Qwen2.5-7B + SimpleRLEvaluation Approach=LLM-as-a-judge (Ours), Model Size=7B, Training Strategy=SimpleRL2026.0442.7
Qwen2.5-7B + SimpleRLEvaluation Approach=Symbolic (Baseline), Model Size=7B, Training Strategy=SimpleRL2026.0441.2
DCRLBackbone=Qwen3-8B-Base2026.0340.9
Qwen2.5-Math-1.5B-Instructnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0940
SFPOModel=Qwen2.5-Math-1.5B2025.1039.72
GRPOModel=Qwen2.5-Math-1.5B2025.1039.42
BaseModel=Qwen2.5-Math-7B2025.1038.54
Qwen2.5-Math-1.5B-Oat-Zeronumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0937.78
DCRLBackbone=Qwen3-4B-Base2026.0337.2
MTTeacher Model=Qwen2.5-Math-7B-Instruct2026.0437
BaseModel=DS-distilled-Qwen-1.5B2025.1036.94
VCRDTeacher Model=Qwen2.5-Math-7B-Instruct, Student Model=Qwen2.5-Math-1.5B2026.0435.4
distillm22026.0434.1
DistillLM-2Teacher Model=Qwen2.5-Math-7B-Instruct, Student Model=Qwen2.5-Math-1.5B2026.0434.1
Qwen2.5-7B (Lighteval)Evaluation Approach=LLM-as-a-judge (Ours), Model Size=7B, Evaluation Framework=Lighteval2026.0433.9
VCRD-Probapproach=PRM-free2026.0433.6
DistilLLMTeacher Model=Qwen2.5-Math-7B-Instruct, Student Model=Qwen2.5-Math-1.5B2026.0433.5
GKDTeacher Model=Qwen2.5-Math-7B-Instruct, Student Model=Qwen2.5-Math-1.5B2026.0433.3
ABKDTeacher Model=Qwen2.5-Math-7B-Instruct, Student Model=Qwen2.5-Math-1.5B2026.0433.2
KDTeacher Model=Qwen2.5-Math-7B-Instruct, Student Model=Qwen2.5-Math-1.5B2026.0432.4
Qwen2.5-14BEvaluation Approach=LLM-as-a-judge (Ours), Model Size=14B2026.0431.4
Qwen2.5-Math-1.5Bnumber of samples=32, model scale=1.5B, evaluation cluster=128×H100 96G2025.0930.74
Qwen2.5-7BEvaluation Approach=LLM-as-a-judge (Ours), Model Size=7B2026.0430.6
Qwen2.5-32BEvaluation Approach=LLM-as-a-judge (Ours), Model Size=32B2026.0430.6
Qwen3-1.7B-BaseTraining Strategy=Full-shot2026.0429.48
Qwen2.5-14BEvaluation Approach=Symbolic (Baseline), Model Size=14B2026.0428.6
BaseModel=Qwen2.5-Math-1.5B2025.1028.45
Qwen2.5-7BEvaluation Approach=Symbolic (Baseline), Model Size=7B2026.0428.1
Qwen2.5-32BEvaluation Approach=Symbolic (Baseline), Model Size=32B2026.0427.9
Qwen2.5-7B (Lighteval)Evaluation Approach=Symbolic (Baseline), Model Size=7B, Evaluation Framework=Lighteval2026.0427.8
Qwen2.5-7B + SimpleRL (Lighteval)Evaluation Approach=Symbolic (Baseline), Model Size=7B, Training Strategy=SimpleRL, Evaluation Framework=Lighteval2026.0427.1
Qwen3-1.7B-BaseTraining Strategy=Only-General2026.0426.07
HEALTraining Strategy=HEAL2026.0425.78
Qwen3-1.7B-BaseTraining Strategy=Hybrid2026.0424.15
Qwen3-1.7B-BaseTraining Strategy=Few-shot2026.0423.7
Qwen3-1.7B-BaseTraining Strategy=Base2026.0421.48
BaseModel=Qwen3-4B-Base2025.1020.66
MSStudent Model=Qwen2.5-Math-1.5B2026.0416.9
DCRLBackbone=Llama3.2-3B-Instruct2026.0315.4
Llama3.1-8BEvaluation Approach=LLM-as-a-judge (Ours), Model Size=8B2026.043.5
Llama3.1-8BEvaluation Approach=Symbolic (Baseline), Model Size=8B2026.042.7