# Mathematical Reasoning on GSM8k, MATH, SVAMP, and MM Source: [Mathematical Reasoning on GSM8k, MATH, SVAMP, and MM]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:45:49.928Z Evaluation of model performance on a suite of mathematical reasoning tasks covering GSM8k, MATH, SVAMP, and MM datasets. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | Mathematical Tasks GSM8k, MATH, SVAMP, MM | | Metric | GSM8k Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 27 of 27 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [GSM8k Accuracy](): Higher is better; unit: PERCENT. - [MATH Accuracy](): Higher is better; unit: PERCENT. - [SVAMP Accuracy](): Higher is better; unit: PERCENT. - [MM Accuracy](): Higher is better; unit: PERCENT. - [Average Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | GSM8k Accuracy | Result date | Report paper | | --- | --- | --- | --- | | + RFT w/ TD | 58.15 | Not recorded | 68fa73b11e209131cea5df03 | | + RLMEC | 58.15 | Not recorded | 68fa73b11e209131cea5df03 | | + RFT w/ RD | 57.39 | Not recorded | 68fa73b11e209131cea5df03 | | + RFT w/ GT | 57.32 | Not recorded | 68fa73b11e209131cea5df03 | | + CoH | 57.31 | Not recorded | 68fa73b11e209131cea5df03 | | SFT LLM | 56.63 | Not recorded | 68fa73b11e209131cea5df03 | | + SFT | 55.88 | Not recorded | 68fa73b11e209131cea5df03 | | + RFT | 55.8 | Not recorded | 68fa73b11e209131cea5df03 | | + Vanilla PPO | 53.45 | Not recorded | 68fa73b11e209131cea5df03 | | + RFT w/ TD | 51.86 | Not recorded | 68fa73b11e209131cea5df03 | | + RFT w/ RD | 51.78 | Not recorded | 68fa73b11e209131cea5df03 | | + RLMEC | 51.18 | Not recorded | 68fa73b11e209131cea5df03 | | SFT LLM | 51.02 | Not recorded | 68fa73b11e209131cea5df03 | | + PPO A2C | 50.94 | Not recorded | 68fa73b11e209131cea5df03 | | + SFT | 50.34 | Not recorded | 68fa73b11e209131cea5df03 | | + CoH | 50.11 | Not recorded | 68fa73b11e209131cea5df03 | | + RFT w/ GT | 49.89 | Not recorded | 68fa73b11e209131cea5df03 | | + RFT | 49.66 | Not recorded | 68fa73b11e209131cea5df03 | | + Vanilla PPO | 48.97 | Not recorded | 68fa73b11e209131cea5df03 | | + DPO | 44.2 | Not recorded | 68fa73b11e209131cea5df03 | | + DPO | 34.19 | Not recorded | 68fa73b11e209131cea5df03 | | WizardLM | 31.01 | Not recorded | 68fa73b11e209131cea5df03 | | Vicuna | 24.1 | Not recorded | 68fa73b11e209131cea5df03 | | LLaMA 2 | 21.23 | Not recorded | 68fa73b11e209131cea5df03 | | WizardLM | 14.48 | Not recorded | 68fa73b11e209131cea5df03 | | Vicuna | 12.2 | Not recorded | 68fa73b11e209131cea5df03 | | LLaMA 2 | 11.22 | Not recorded | 68fa73b11e209131cea5df03 | ### + RFT w/ TD GSM8k Accuracy: 58.15. Result ID: 6993182c3a6c31484913abbc. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RLMEC GSM8k Accuracy: 58.15. Result ID: 6993182c3a6c31484913abc1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RFT w/ RD GSM8k Accuracy: 57.39. Result ID: 6993182c3a6c31484913abbd. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RFT w/ GT GSM8k Accuracy: 57.32. Result ID: 6993182c3a6c31484913abbb. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + CoH GSM8k Accuracy: 57.31. Result ID: 6993182c3a6c31484913abbe. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### SFT LLM GSM8k Accuracy: 56.63. Result ID: 6993182c3a6c31484913abb8. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"13B"}. This record does not assert independent reproduction. ### + SFT GSM8k Accuracy: 55.88. Result ID: 6993182c3a6c31484913abb9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RFT GSM8k Accuracy: 55.8. Result ID: 6993182c3a6c31484913abba. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + Vanilla PPO GSM8k Accuracy: 53.45. Result ID: 6993182c3a6c31484913abc0. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RFT w/ TD GSM8k Accuracy: 51.86. Result ID: 6993182c3a6c31484913abae. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RFT w/ RD GSM8k Accuracy: 51.78. Result ID: 6993182c3a6c31484913abaf. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RLMEC GSM8k Accuracy: 51.18. Result ID: 6993182c3a6c31484913abb4. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### SFT LLM GSM8k Accuracy: 51.02. Result ID: 6993182c3a6c31484913abaa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"7B"}. This record does not assert independent reproduction. ### + PPO A2C GSM8k Accuracy: 50.94. Result ID: 6993182c3a6c31484913abb3. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + SFT GSM8k Accuracy: 50.34. Result ID: 6993182c3a6c31484913abab. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + CoH GSM8k Accuracy: 50.11. Result ID: 6993182c3a6c31484913abb0. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RFT w/ GT GSM8k Accuracy: 49.89. Result ID: 6993182c3a6c31484913abad. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + RFT GSM8k Accuracy: 49.66. Result ID: 6993182c3a6c31484913abac. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + Vanilla PPO GSM8k Accuracy: 48.97. Result ID: 6993182c3a6c31484913abb2. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + DPO GSM8k Accuracy: 44.2. Result ID: 6993182c3a6c31484913abbf. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### + DPO GSM8k Accuracy: 34.19. Result ID: 6993182c3a6c31484913abb1. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B","Base Model":"SFT LLM"}. This record does not assert independent reproduction. ### WizardLM GSM8k Accuracy: 31.01. Result ID: 6993182c3a6c31484913abb7. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B"}. This record does not assert independent reproduction. ### Vicuna GSM8k Accuracy: 24.1. Result ID: 6993182c3a6c31484913abb6. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B"}. This record does not assert independent reproduction. ### LLaMA 2 GSM8k Accuracy: 21.23. Result ID: 6993182c3a6c31484913abb5. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"13B"}. This record does not assert independent reproduction. ### WizardLM GSM8k Accuracy: 14.48. Result ID: 6993182c3a6c31484913aba9. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B"}. This record does not assert independent reproduction. ### Vicuna GSM8k Accuracy: 12.2. Result ID: 6993182c3a6c31484913aba8. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B"}. This record does not assert independent reproduction. ### LLaMA 2 GSM8k Accuracy: 11.22. Result ID: 6993182c3a6c31484913aba7. [Reporting paper]() Recorded evaluation configuration: {"Model Parameters":"7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.