Mathematical Reasoning on Minerva Math (Accuracy, Pass@3)
44.73Accuracy (Avg)GRAIL
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GRAILModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 44.73 | 48.89 | |
| GRAILModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 44.36 | 47.79 | |
| GRPOModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 41.42 | 45.96 | |
| GRPOModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 39.83 | 47.06 | |
| GRAILModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 30.88 | 41.54 | |
| GRPOModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 30.64 | 42.28 | |
| GRAILModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 25.12 | 36.4 | |
| GRPOModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 21.08 | 31.99 | |
| GRAILModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 16.05 | 24.63 | |
| GRPOModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 14.95 | 24.63 |