Mathematical Reasoning on AMC 23 (Average Accuracy, Pass@3)
82.5Average AccuracyGRAIL
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GRAILModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 82.5 | 87.5 | |
| GRAILModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 81.67 | 90 | |
| GRAILModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 80 | 97.5 | |
| GRPOModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 77.5 | 92.5 | |
| GRPOModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 77.5 | 90 | |
| GRPOModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 71.67 | 85 | |
| GRAILModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 26.67 | 40 | |
| GRAILModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 23.33 | 40 | |
| GRPOModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 22.5 | 32.5 | |
| GRPOModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 12.5 | 22.5 |