Mathematical Reasoning on Olympiad Bench (Average Accuracy, Pass@3)
49.17Average AccuracyGRAIL
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GRAILModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 49.17 | 53 | |
| GRAILModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 47.5 | 54 | |
| GRPOModel=Qwen3-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 46.67 | 52.5 | |
| GRPOModel=Qwen3-4B, Sampling Temperature=0.6, Generations per problem=32026.06 | 45.83 | 53.5 | |
| GRPOModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 42.17 | 48.5 | |
| GRAILModel=R1-Distill-Llama-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 42 | 50 | |
| GRAILModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 20.5 | 29.5 | |
| GRAILModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 16 | 27 | |
| GRPOModel=OctoThinker-3B, Sampling Temperature=0.6, Generations per problem=32026.06 | 15 | 26 | |
| GRPOModel=OctoThinker-8B, Sampling Temperature=0.6, Generations per problem=32026.06 | 14.67 | 25.5 |