List your company
ResearchFormerly WizWand

Mathematical Reasoning on AMC23 (PASS@1 Accuracy and Output Length)

100PASS@1 AccuracyTIP
61.5271.5181.591.49Aug 8, 2025Sep 26, 2025Nov 15, 2025Jan 4, 2026Feb 23, 2026Apr 14, 2026Jun 3, 2026
Updated 4mo ago

Evaluation results

MethodLinks
TIPBase Model=QwQ-32B2026.051006,454—
CyclicReflexBase Model=QwQ-32B2026.051007,076—
PathCalBase Model=QwQ-32B2026.051006,608—
Qwen3-4B (Ours)Backbone=Qwen3-4B2026.0497.5——
S1Base Model=QwQ-32B2026.0597.57,295—
OriginalBase Model=QwQ-32B2026.05957,524—
DS-Llama-8B (Ours)Backbone=DS-Llama-8B2026.0492.5——
Pass@k†Backbone=Qwen2.5-7B-Instruct, N=322026.0590.527,979—
PathCalBase Model=DeepSeek-R1-Distill-Qwen-14B2026.05903,166—
PieceHint-Nemotron-1.5BModel Name=Nemotron-1.5B, Model Size=1.5B, Method Type=PieceHint2026.0489.1——
DeepSeek-R1-Distill-32BModel Name=DeepSeek-R1-Distill-32B, Model Size=32B, Method Type=Baseline2026.0487.3——
CyclicReflexBase Model=DeepSeek-R1-Distill-Qwen-14B2026.05853,695—
Pass@kk=16, Model=Qwen2.5-7B2026.0684.2——
MixedFramework=TD-Grokking2026.0683.33——
LaserBackbone=DeepSeek-R1-Distill-Qwen-7B2026.0583.13,388.5—
Nemotron-1.5BModel Name=Nemotron-1.5B, Model Size=1.5B, Method Type=Baseline2026.0482.7——
CoDModel=DeepSeek-R1-Distill-Qwen-7B2026.0482.53,2172.56
TIPBase Model=DeepSeek-R1-Distill-Qwen-7B2026.0582.52,893—
S1Base Model=DeepSeek-R1-Distill-Qwen-7B2026.0582.53,335—
PathCalBase Model=DeepSeek-R1-Distill-Qwen-7B2026.0582.52,244—
OriginalBase Model=DeepSeek-R1-Distill-Qwen-14B2026.0582.53,733—
TIPBase Model=DeepSeek-R1-Distill-Qwen-14B2026.0582.53,733—
ExpThinkBackbone=Qwen3-8B2026.0582.42,232—
JETBackbone=DeepSeek-R1-Distill-Qwen-7B2026.0582.25,573.4—
AutoThinkBackbone=DeepSeek-R1-Distill-Qwen-7B2026.0581.64,579.3—
DynamicFramework=TD-Grokking2026.0680.83——
TRiMSBackbone=Qwen3-4B2026.0380.71,046.2—
PieceHint-Qwen3-1.7BModel Name=Qwen3-1.7B, Model Size=1.7B, Method Type=PieceHint2026.0480.4——
ExpThinkBackbone=DeepSeek-R1-Distill-Qwen-7B2026.0580.42,775.1—
Qwen3-4BModel Name=Qwen3-4B, Model Size=4B, Method Type=Baseline2026.0480——
CyclicReflexBase Model=DeepSeek-R1-Distill-Qwen-7B2026.05802,806—
Sub-onlyFramework=TD-Grokking2026.0679.58——
AdaptThinkBackbone=DeepSeek-R1-Distill-Qwen-7B2026.0579.14,754.1—
Pass@k†Backbone=Phi-4-mini-Instruct, N=322026.057925,003—
VanillaBackbone=DeepSeek-R1-Distill-Qwen-7B2026.0578.96,907.2—
DeepSeek-R1-Distill-7BModel Name=DeepSeek-R1-Distill-7B, Model Size=7B, Method Type=Baseline2026.0478.8——
TALEModel=DeepSeek-R1-Distill-Qwen-7B2026.0477.54,1281.73
CRISPModel=DeepSeek-R1-Distill-Qwen-7B2026.0477.52,1803.56
GRPO + Gaussian ReweightFine-tuning Samples=7000, Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=zero-shot pass@12026.0577.5——
OriginalBase Model=DeepSeek-R1-Distill-Qwen-7B2026.0577.52,750—
S1Base Model=DeepSeek-R1-Distill-Qwen-14B2026.0577.53,980—
TIPBase Model=DeepSeek-R1-Distill-Llama-8B2026.0577.53,310—
VanillaFramework=GRPO2026.0677.5——
QuestAFramework=GRPO2026.0677.5——
VanillaBackbone=Qwen3-8B2026.0577.49,139.1—
VanillaBackbone=Qwen3-4B2026.0377.19,060—
LC-R1Backbone=DeepSeek-R1-Distill-Qwen-7B2026.0576.43,678.1—
Scaf-GRPOFramework=GRPO2026.0676.25——
OriginalBase Model=DeepSeek-R1-Distill-Llama-8B2026.05753,937—
CyclicReflexBase Model=DeepSeek-R1-Distill-Llama-8B2026.05753,937—
PathCalBase Model=DeepSeek-R1-Distill-Llama-8B2026.05753,064—
AblationFramework=GRPO2026.0675——
SFTFramework=GRPO2026.0675——
Base2026.0673.75——
DeepScaleR-1.5B-PreviewFine-tuning Samples=40,000, Evaluation Protocol=zero-shot pass@12026.0573.6——
LEADBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Max Response Length=8K2026.0573.54,368—
OriginalModel=DeepSeek-R1-Distill-Qwen-7B2026.0472.54,4831.62
TokenSkipModel=DeepSeek-R1-Distill-Qwen-7B2026.0472.53,6761.97
LUFFYFine-tuning base=Qwen2.5-Math-7B, Comparison Group=Non-comparable Baselines2025.0872.5——
S1Base Model=DeepSeek-R1-Distill-Llama-8B2026.0572.54,291—
Re-Schedule_linearBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@32, Weighting Scheme=linear2025.1072.4——
Re-Schedule_sigmoidBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@32, Weighting Scheme=sigmoid2025.1072.3——
OPOBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@322025.1071.5——
GRPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Max Response Length=8K2026.0571.52,607—
PieceHint-DeepSeek-1.5BModel Name=DeepSeek-R1-Distill-1.5B, Model Size=1.5B, Method Type=PieceHint2026.0471.2——
Seed-GRPOBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@322025.1071——
ACC_sigmoidBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@32, Weighting Scheme=sigmoid2025.1070.9——
DRPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Max Response Length=8K2026.05703,665—
Open-RS1Fine-tuning Samples=18,615, Evaluation Protocol=zero-shot pass@12026.0570——
Clip-CovFine-tuning Samples=7000, Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=zero-shot pass@12026.0570——
BaseBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Max Response Length=8K2026.0569.257,472—
Re-Schedule_sigmoidBackbone=Qwen2.5-7B, Weight Function=sigmoid2025.1069.2——
Qwen3-1.7BModel Name=Qwen3-1.7B, Model Size=1.7B, Method Type=Baseline2026.0469——
ACC_sigmoidBackbone=Qwen2.5-7B, Category=Scheduling Methods2025.1068.6——
Re-Schedule_linearBackbone=Qwen2.5-7B, Weight Function=linear2025.1068.6——
GRPOFine-tuning Samples=7000, Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=zero-shot pass@12026.0567.5——
SPSModel=Qwen2.5-7B-Instruct, N=82026.0567.522,446—
SPSBackbone=Qwen2.5-7B-Instruct, N=82026.0567.522,446—
JETBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.0567.25,671.4—
GDPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Max Response Length=8K2026.05672,795—
Still-3-1.5B-PreviewFine-tuning Samples=30,000, Evaluation Protocol=zero-shot pass@12026.0566.7——
PB-SMCModel=Qwen2.5-7B-Instruct, N=82026.0566.517,637—
PB-SMCBackbone=Qwen2.5-7B-Instruct, N=82026.0566.517,637—
GRPOBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@322025.1066.2——
ExpThinkBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.0565.82,921.2—
CGSk=16, L=20, Base Model=Qwen2.5-7B, Guidance Model=Qwen2.5-72B2026.0665.8——
SimpleRL-ZooBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@322025.1065.4——
Eurus-PRIMEBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@322025.1065.2——
TruncationModel=DeepSeek-R1-Distill-Qwen-7B2026.04653,0352.14
Best-of-NBackbone=Qwen2.5-7B-Instruct, N=322026.056527,564—
Large greedyModel=Qwen2.5-72B2026.0665——
PRM-72B guided searchk=16, L=20, Base Model=Qwen2.5-7B, Guidance Model=Qwen2.5-72B2026.0665——
SimpleRL-ZooBackbone=Qwen2.5-7B, Category=Classical RLVR Methods2025.1064.7——
OPOBackbone=Qwen2.5-7B, Category=Classical RLVR Methods2025.1064.6——
LPPOBackbone=Qwen2.5-7B, Category=Scheduling Methods2025.1064——
DeepSeek-R1-Distill-1.5BModel Name=DeepSeek-R1-Distill-1.5B, Model Size=1.5B, Method Type=Baseline2026.0463.6——
LPPOBackbone=Qwen2.5-Math-7B, Evaluation Protocol=avg@322025.1063.3——
Seed-GRPOBackbone=Qwen2.5-7B, Category=Scheduling Methods2025.1063.3——
SPSModel=Qwen2.5-3B-Instruct, N=82026.056316,644—
DVTSBackbone=Qwen2.5-7B-Instruct, N=322026.056334,670—