List your company
ResearchFormerly WizWand

Mathematical Reasoning on Minerva Math v1 (test)

57.4Accuracy (avg@1)Pass@k
4.98418.59232.245.808May 23, 2025Jul 24, 2025Sep 24, 2025Nov 26, 2025Jan 27, 2026Mar 30, 2026Jun 1, 2026
Updated 5mo ago

Evaluation results

MethodLinks
Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0657.4
Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0650.7
Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0650
Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0650
Large greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0650
NFTBackbone=Qwen2.5-32B, Fine-tuning Protocol=Supervised fine-tuning2025.0548.9
DAPOBackbone=Qwen2.5-32B, Fine-tuning Protocol=Reinforcement fine-tuning2025.0547.5
Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.0646
Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.0646
Large greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0646
RFTBackbone=Qwen2.5-32B, Fine-tuning Protocol=Supervised fine-tuning2025.0544.1
NEMOTRON-NANO-12B-V2 (RLP + Post)Pre-training objective=RLP, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.0942.78
NFTBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Supervised fine-tuning2025.0540.8
NEMOTRON-NANO-12B-V2 (Base + Post)Pre-training objective=Next-token prediction, Post-training pipeline=SFT + RLVR, Evaluation protocol=Pass@1 (average of 8 runs)2025.0940.76
NEMOTRON-NANO-12B-V2 (RLP)Pre-training objective=RLP, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.0939.48
DAPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Reinforcement fine-tuning2025.0539.3
DPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Preference fine-tuning2025.0539
CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.0639
RFTBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Supervised fine-tuning2025.0538.6
GRPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Reinforcement fine-tuning2025.0538.2
Dr. GRPOBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Reinforcement fine-tuning2025.0538.2
Pass@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0638.2
CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.0636.8
LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=32, Chunk Length (L)=202026.0636
Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0635.3
LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=16, Chunk Length (L)=202026.0633.8
CGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.0632.7
PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0632.4
LGSModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=8, Chunk Length (L)=202026.0631.6
Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.0629.4
PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0628.3
Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0628.3
Qwen2.5-32BBackbone=Qwen2.5-32B, Fine-tuning Protocol=Base2025.0527.9
Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0626.8
Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0626.5
Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0626.1
Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0626.1
Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0625.7
PRM guided searchModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0625
Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0624.3
Borda count (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0622.8
NEMOTRON-NANO-12B-V2 (Base)Pre-training objective=Next-token prediction, Post-training pipeline=None, Evaluation protocol=Pass@1 (average of 8 runs)2025.0922.61
Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0622.4
Borda count (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0622.1
CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.0621.7
Qwen2.5-Math-7BBackbone=Qwen2.5-Math-7B, Fine-tuning Protocol=Base2025.0521.3
Majority@kModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0621.3
Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0621.3
Pass@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.0621
Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0620.6
Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0620.6
Self-Certainty (large)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0619.9
Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0619.5
Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0619.1
Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=162026.0619.1
Small greedyModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=322026.0619.1
Self-Certainty (small)Model Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0618
CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.0616.2
LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.0616.2
Best-of-NModel Pair=Qwen2.5-1.5B → Qwen2.5-32B, Sample Size (k)=82026.0615.8
LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=32, Chunk Length (L)=202026.0615.4
CGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=16, Chunk Length (L)=202026.0615.1
PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.0613.2
LGSModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=8, Chunk Length (L)=202026.0612.9
PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0612.5
Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0611.8
Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0611.4
PRM guided searchModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.0611
Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.0610.7
Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.069.9
Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.069.6
Self-Certainty (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.069.6
Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.069.6
Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.069.6
Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.069.2
Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.069.2
Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.068.8
Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.068.5
Borda count (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.068.5
Borda count (large)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.068.5
Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.068.1
Best-of-NModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.067.7
Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=82026.067.4
Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.067.4
Small greedyModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.067.4
Majority@kModel Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=322026.067.4
Self-Certainty (small)Model Pair=Llama-3.2-1B → Llama-3.1-70B, Sample Size (k)=162026.067