Mathematical Reasoning on MATH500 (Average Accuracy, Pass@3)
92.07Average Accuracy (MATH500)GRAIL
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRAILModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 92.07 | 95.4 | |
| GRAILModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 91.47 | 93.8 | |
| GRPOModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 90.07 | 95 | |
| GRPOModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 88.07 | 93.6 | |
| GRAILModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 82.6 | 91 | |
| GRPOModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 81 | 89 | |
| GRAILModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 54.53 | 68.4 | |
| GRPOModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 48.53 | 65.2 | |
| GRAILModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 46.4 | 61.4 | |
| GRPOModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 43.87 | 59.6 |