# Mathematical Reasoning on CMath (pass@1) Source: [Mathematical Reasoning on CMath (pass@1)]() Collection index: [llms.txt]() Data updated: 2026-05-27T08:18:43.769Z Evaluation of model performance on mathematical reasoning problems using the CMath dataset, measured by pass@1 score. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | CMath | | Metric | Pass@1 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 31 of 31 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 | Result date | Report paper | | --- | --- | --- | --- | | Qwen2.5-7B + GRPO | 91.7 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen2.5-7B + GRPO w/ VERL. | 90.7 | Not recorded | 68fa7bc11e209131cea8a35b | | Qwen2.5-7B + PPO w/ VERL. | 90.3 | Not recorded | 68fa7bc11e209131cea8a35b | | VCRD | 90.2 | Not recorded | 69fc1c5e5497b3b670c88337 | | Qwen2.5-7B + PPO | 89.8 | Not recorded | 68fa7bc11e209131cea8a35b | | MT | 89.8 | Not recorded | 69fc1c5e5497b3b670c88337 | | DistillLM-2 | 89.7 | Not recorded | 69fc1c5e5497b3b670c88337 | | DistilLLM | 89.3 | Not recorded | 69fc1c5e5497b3b670c88337 | | GKD | 88.8 | Not recorded | 69fc1c5e5497b3b670c88337 | | ABKD | 88.5 | Not recorded | 69fc1c5e5497b3b670c88337 | | KD | 88.2 | Not recorded | 69fc1c5e5497b3b670c88337 | | Qwen2.5-7B | 86.7 | Not recorded | 68fa7bc11e209131cea8a35b | | GPT-4 | 86 | Not recorded | 68fa70e71e209131cea4dd44 | | KwaiYiiMath-HPA | 85.83 | Not recorded | 68fa70e71e209131cea4dd44 | | KwaiYiiMath | 85.33 | Not recorded | 68fa70e71e209131cea4dd44 | | Ernie Bot | 84.33 | Not recorded | 68fa70e71e209131cea4dd44 | | ChatGPT | 73.83 | Not recorded | 68fa70e71e209131cea4dd44 | | ChatGLM2 | 68.36 | Not recorded | 68fa70e71e209131cea4dd44 | | Qwen2.5-Dense | 63.99 | Not recorded | 6a167aafdbd7150506f800fc | | QWen | 63.16 | Not recorded | 68fa70e71e209131cea4dd44 | | MS | 62 | Not recorded | 69fc1c5e5497b3b670c88337 | | Qwen2.5-Dense2MoE | 55.5 | Not recorded | 6a167aafdbd7150506f800fc | | BaiChuan1 | 51.33 | Not recorded | 68fa70e71e209131cea4dd44 | | WizardMath | 50.83 | Not recorded | 68fa70e71e209131cea4dd44 | | Llama-3.2-3B-Instruct + GRPO w/ VERL. | 46.2 | Not recorded | 68fa7bc11e209131cea8a35b | | Llama-3.2-3B-Instruct + PPO w/ VERL. | 30.7 | Not recorded | 68fa7bc11e209131cea8a35b | | Llama-3.2-3B-Instruct + GRPO | 28.3 | Not recorded | 68fa7bc11e209131cea8a35b | | Llama2-Dense2MoE | 25.33 | Not recorded | 6a167aafdbd7150506f800fc | | Llama-3.2-3B-Instruct + PPO | 21.2 | Not recorded | 68fa7bc11e209131cea8a35b | | Llama2-Dense | 16.6 | Not recorded | 6a167aafdbd7150506f800fc | | Llama-3.2-3B-Instruct | 10.2 | Not recorded | 68fa7bc11e209131cea8a35b | ### Qwen2.5-7B + GRPO Pass@1: 91.7. Result ID: 69ddfb2047f34ce924c56161. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","RL Framework":"GRPO","VERL Integration":"false"}. This record does not assert independent reproduction. ### Qwen2.5-7B + GRPO w/ VERL. Pass@1: 90.7. Result ID: 69ddfb2047f34ce924c56162. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","RL Framework":"GRPO","VERL Integration":"true"}. This record does not assert independent reproduction. ### Qwen2.5-7B + PPO w/ VERL. Pass@1: 90.3. Result ID: 69ddfb2047f34ce924c56164. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","RL Framework":"PPO","VERL Integration":"true"}. This record does not assert independent reproduction. ### VCRD Pass@1: 90.2. Result ID: 69fc3f5c47f34ce924de9c67. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher Model":"Qwen2.5-Math-7B-Instruct","Student Model":"Qwen2.5-Math-1.5B"}. This record does not assert independent reproduction. ### Qwen2.5-7B + PPO Pass@1: 89.8. Result ID: 69ddfb2047f34ce924c56163. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","RL Framework":"PPO","VERL Integration":"false"}. This record does not assert independent reproduction. ### MT Pass@1: 89.8. Result ID: 69fc3f5c47f34ce924de9c60. [Reporting paper]() Recorded evaluation configuration: {"Teacher Model":"Qwen2.5-Math-7B-Instruct"}. This record does not assert independent reproduction. ### DistillLM-2 Pass@1: 89.7. Result ID: 69fc3f5c47f34ce924de9c66. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher Model":"Qwen2.5-Math-7B-Instruct","Student Model":"Qwen2.5-Math-1.5B"}. This record does not assert independent reproduction. ### DistilLLM Pass@1: 89.3. Result ID: 69fc3f5c47f34ce924de9c64. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher Model":"Qwen2.5-Math-7B-Instruct","Student Model":"Qwen2.5-Math-1.5B"}. This record does not assert independent reproduction. ### GKD Pass@1: 88.8. Result ID: 69fc3f5c47f34ce924de9c63. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher Model":"Qwen2.5-Math-7B-Instruct","Student Model":"Qwen2.5-Math-1.5B"}. This record does not assert independent reproduction. ### ABKD Pass@1: 88.5. Result ID: 69fc3f5c47f34ce924de9c65. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher Model":"Qwen2.5-Math-7B-Instruct","Student Model":"Qwen2.5-Math-1.5B"}. This record does not assert independent reproduction. ### KD Pass@1: 88.2. Result ID: 69fc3f5c47f34ce924de9c62. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Teacher Model":"Qwen2.5-Math-7B-Instruct","Student Model":"Qwen2.5-Math-1.5B"}. This record does not assert independent reproduction. ### Qwen2.5-7B Pass@1: 86.7. Result ID: 69ddfb2047f34ce924c56160. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Qwen2.5-7B","RL Framework":"None","VERL Integration":"false"}. This record does not assert independent reproduction. ### GPT-4 Pass@1: 86. Result ID: 69917c913a6c314849087a50. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### KwaiYiiMath-HPA Pass@1: 85.83. Result ID: 69917c913a6c314849087a58. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"13B","Human preference alignment":true}. This record does not assert independent reproduction. ### KwaiYiiMath Pass@1: 85.33. Result ID: 69917c913a6c314849087a57. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"13B","Human preference alignment":false}. This record does not assert independent reproduction. ### Ernie Bot Pass@1: 84.33. Result ID: 69917c913a6c314849087a52. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### ChatGPT Pass@1: 73.83. Result ID: 69917c913a6c314849087a51. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### ChatGLM2 Pass@1: 68.36. Result ID: 69917c913a6c314849087a55. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"6B"}. This record does not assert independent reproduction. ### Qwen2.5-Dense Pass@1: 63.99. Result ID: 6a16a8ddfbc373592c0fbbcd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Qwen2.5","Model":"Dense","Activated Parameters":"1.5 B"}. This record does not assert independent reproduction. ### QWen Pass@1: 63.16. Result ID: 69917c913a6c314849087a56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"7B"}. This record does not assert independent reproduction. ### MS Pass@1: 62. Result ID: 69fc3f5c47f34ce924de9c61. [Reporting paper]() Recorded evaluation configuration: {"Student Model":"Qwen2.5-Math-1.5B"}. This record does not assert independent reproduction. ### Qwen2.5-Dense2MoE Pass@1: 55.5. Result ID: 6a16a8ddfbc373592c0fbbce. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Qwen2.5","Model":"Ours","Activated Parameters":"1.2 B"}. This record does not assert independent reproduction. ### BaiChuan1 Pass@1: 51.33. Result ID: 69917c913a6c314849087a53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"#Params":"13B"}. This record does not assert independent reproduction. ### WizardMath Pass@1: 50.83. Result ID: 69917c913a6c314849087a54. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"#Params":"13B"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + GRPO w/ VERL. Pass@1: 46.2. Result ID: 69ddfb2047f34ce924c5615d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","RL Framework":"GRPO","VERL Integration":"true"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + PPO w/ VERL. Pass@1: 30.7. Result ID: 69ddfb2047f34ce924c5615f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","RL Framework":"PPO","VERL Integration":"true"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + GRPO Pass@1: 28.3. Result ID: 69ddfb2047f34ce924c5615c. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","RL Framework":"GRPO","VERL Integration":"false"}. This record does not assert independent reproduction. ### Llama2-Dense2MoE Pass@1: 25.33. Result ID: 6a16a8ddfbc373592c0fbbd0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"LLM":"Llama2","Model":"Ours","Activated Parameters":"5.7 B"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct + PPO Pass@1: 21.2. Result ID: 69ddfb2047f34ce924c5615e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","RL Framework":"PPO","VERL Integration":"false"}. This record does not assert independent reproduction. ### Llama2-Dense Pass@1: 16.6. Result ID: 6a16a8ddfbc373592c0fbbcf. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"LLM":"Llama2","Model":"Dense","Activated Parameters":"7.0 B"}. This record does not assert independent reproduction. ### Llama-3.2-3B-Instruct Pass@1: 10.2. Result ID: 69ddfb2047f34ce924c5615b. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B-Instruct","RL Framework":"None","VERL Integration":"false"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.