Mathematical Reasoning on AIME24 (Average Accuracy, Pass@3)
47.78Average AccuracyGRAIL
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GRAILModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 47.78 | 60 | |
| GRAILModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 36.67 | 53.33 | |
| GRPOModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 36.67 | 50 | |
| GRPOModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 27.78 | 43.33 | |
| GRAILModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 25.56 | 36.67 | |
| GRPOModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 21.11 | 36.67 | |
| GRAILModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 4.44 | 10 | |
| GRAILModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 3.33 | 6.67 | |
| GRPOModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 2.22 | 3.33 | |
| GRPOModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 1.11 | 3.33 |