# Mathematical Reasoning on Olympiad-Bench (Pass@1/Pass@8 Correctness and Length) Source: [Mathematical Reasoning on Olympiad-Bench (Pass@1/Pass@8 Correctness and Length)]() Collection index: [llms.txt]() Data updated: 2026-05-11T07:10:16.009Z Evaluation of model performance on mathematical reasoning problems using the Olympiad-Bench dataset, measuring both solution correctness (Pass@1, Pass@8) and the average length of the generated solutions. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | Olympiad-Bench | | Metric | Accuracy Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy Pass@1](): Higher is better; unit: PERCENT. - [Accuracy Pass@8](): Higher is better; unit: PERCENT. - [Average Length Pass@1](): Lower is better; unit: SIZE. - [Average Length Pass@8](): Lower is better; unit: SIZE. ## Recorded results | Method | Accuracy Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | CoT Cold-Start + R-Zero | 46.88 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + Solver Feedback | 46.59 | Not recorded | 6916b7990052cac1dc65e123 | | CoT Cold-Start + Solver Feedback | 46.59 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct | 45.4 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + R-Zero | 44.81 | Not recorded | 6916b7990052cac1dc65e123 | | Seed Set | 44.51 | Not recorded | 6916b7990052cac1dc65e123 | | CoT Cold-Start + Solver Feedback | 44.36 | Not recorded | 6916b7990052cac1dc65e123 | | CoT Cold-Start + RLMT | 44.21 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + RLMT | 44.15 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + R-Zero | 44.07 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + Solver Feedback | 43.47 | Not recorded | 6916b7990052cac1dc65e123 | | CoT Cold-Start | 41.39 | Not recorded | 6916b7990052cac1dc65e123 | | CoT Cold-Start | 41.25 | Not recorded | 6916b7990052cac1dc65e123 | | CoT Cold-Start + RLMT | 40.5 | Not recorded | 6916b7990052cac1dc65e123 | | CoT Cold-Start + R-Zero | 39.91 | Not recorded | 6916b7990052cac1dc65e123 | | Seed Set | 39.76 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + CoT-Self-Instruct | 39.61 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + RLMT | 39.26 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct + CoT-Self-Instruct | 38.58 | Not recorded | 6916b7990052cac1dc65e123 | | Self-Instruct | 37.69 | Not recorded | 6916b7990052cac1dc65e123 | | Base Model | 34.42 | Not recorded | 6916b7990052cac1dc65e123 | | Base Model | 31.01 | Not recorded | 6916b7990052cac1dc65e123 | | DAPO w LATR | 30.4 | Not recorded | 69019f783ef7ccd74e5e39f0 | | GRPO w LATR | 29.5 | Not recorded | 69019f783ef7ccd74e5e39f0 | | DAPO w Stoch. | 28.1 | Not recorded | 69019f783ef7ccd74e5e39f0 | | GRPO w Stoch. | 27.2 | Not recorded | 69019f783ef7ccd74e5e39f0 | | Qwen2.5-3B | 8.5 | Not recorded | 69019f783ef7ccd74e5e39f0 | ### CoT Cold-Start + R-Zero Accuracy Pass@1: 46.88. Result ID: 6a0180c547f34ce924e2dcfb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"6434"}. This record does not assert independent reproduction. ### Self-Instruct + Solver Feedback Accuracy Pass@1: 46.59. Result ID: 6a0180c547f34ce924e2dcf8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"6333"}. This record does not assert independent reproduction. ### CoT Cold-Start + Solver Feedback Accuracy Pass@1: 46.59. Result ID: 6a0180c547f34ce924e2dcfc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"6801"}. This record does not assert independent reproduction. ### Self-Instruct Accuracy Pass@1: 45.4. Result ID: 6a0180c547f34ce924e2dcf4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"5990"}. This record does not assert independent reproduction. ### Self-Instruct + R-Zero Accuracy Pass@1: 44.81. Result ID: 6a0180c547f34ce924e2dcec. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"6461"}. This record does not assert independent reproduction. ### Seed Set Accuracy Pass@1: 44.51. Result ID: 6a0180c547f34ce924e2dcf3. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"4000"}. This record does not assert independent reproduction. ### CoT Cold-Start + Solver Feedback Accuracy Pass@1: 44.36. Result ID: 6a0180c547f34ce924e2dcf1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"6722"}. This record does not assert independent reproduction. ### CoT Cold-Start + RLMT Accuracy Pass@1: 44.21. Result ID: 6a0180c547f34ce924e2dcfa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"6382"}. This record does not assert independent reproduction. ### Self-Instruct + RLMT Accuracy Pass@1: 44.15. Result ID: 6a0180c547f34ce924e2dcf6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"7128"}. This record does not assert independent reproduction. ### Self-Instruct + R-Zero Accuracy Pass@1: 44.07. Result ID: 6a0180c547f34ce924e2dcf7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"7142"}. This record does not assert independent reproduction. ### Self-Instruct + Solver Feedback Accuracy Pass@1: 43.47. Result ID: 6a0180c547f34ce924e2dced. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"5489"}. This record does not assert independent reproduction. ### CoT Cold-Start Accuracy Pass@1: 41.39. Result ID: 6a0180c547f34ce924e2dcf9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"6723"}. This record does not assert independent reproduction. ### CoT Cold-Start Accuracy Pass@1: 41.25. Result ID: 6a0180c547f34ce924e2dcee. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"6616"}. This record does not assert independent reproduction. ### CoT Cold-Start + RLMT Accuracy Pass@1: 40.5. Result ID: 6a0180c547f34ce924e2dcef. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"6703"}. This record does not assert independent reproduction. ### CoT Cold-Start + R-Zero Accuracy Pass@1: 39.91. Result ID: 6a0180c547f34ce924e2dcf0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"6865"}. This record does not assert independent reproduction. ### Seed Set Accuracy Pass@1: 39.76. Result ID: 6a0180c547f34ce924e2dce8. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"4000"}. This record does not assert independent reproduction. ### Self-Instruct + CoT-Self-Instruct Accuracy Pass@1: 39.61. Result ID: 6a0180c547f34ce924e2dcf5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"6388"}. This record does not assert independent reproduction. ### Self-Instruct + RLMT Accuracy Pass@1: 39.26. Result ID: 6a0180c547f34ce924e2dceb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"6859"}. This record does not assert independent reproduction. ### Self-Instruct + CoT-Self-Instruct Accuracy Pass@1: 38.58. Result ID: 6a0180c547f34ce924e2dcea. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"7051"}. This record does not assert independent reproduction. ### Self-Instruct Accuracy Pass@1: 37.69. Result ID: 6a0180c547f34ce924e2dce9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"5375"}. This record does not assert independent reproduction. ### Base Model Accuracy Pass@1: 34.42. Result ID: 6a0180c547f34ce924e2dcf2. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","# Train":"-"}. This record does not assert independent reproduction. ### Base Model Accuracy Pass@1: 31.01. Result ID: 6a0180c547f34ce924e2dce7. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Base","# Train":"-"}. This record does not assert independent reproduction. ### DAPO w LATR Accuracy Pass@1: 30.4. Result ID: 69a7ef776dbb7075b94cd0dd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B","RL Algorithm":"DAPO","Rollout Strategy":"LATR"}. This record does not assert independent reproduction. ### GRPO w LATR Accuracy Pass@1: 29.5. Result ID: 69a7ef776dbb7075b94cd0dc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B","RL Algorithm":"GRPO","Rollout Strategy":"LATR"}. This record does not assert independent reproduction. ### DAPO w Stoch. Accuracy Pass@1: 28.1. Result ID: 69a7ef776dbb7075b94cd0db. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B","RL Algorithm":"DAPO","Rollout Strategy":"Stochastic Sampling"}. This record does not assert independent reproduction. ### GRPO w Stoch. Accuracy Pass@1: 27.2. Result ID: 69a7ef776dbb7075b94cd0da. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B","RL Algorithm":"GRPO","Rollout Strategy":"Stochastic Sampling"}. This record does not assert independent reproduction. ### Qwen2.5-3B Accuracy Pass@1: 8.5. Result ID: 69a7ef776dbb7075b94cd0d9. [Reporting paper]() Recorded evaluation configuration: {"Model":"Qwen2.5-3B","Configuration":"Base Model"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.