# Mathematical Reasoning on MWPBENCH (out-of-domain)

Source: [Mathematical Reasoning on MWPBENCH (out-of-domain)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:39:31.049Z

Evaluation of model performance on mathematical reasoning tasks aggregated under the MWPBENCH (out-of-domain) suite, covering various sub-benchmarks like College Math, TAL Math, and AGIE components.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | MWPBENCH (out-of-domain) |
| Metric | College Math |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 18 of 18 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [College Math](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=College%20Math>): Higher is better; unit: PERCENT.

- [TAL Math](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=TAL%20Math>): Higher is better; unit: PERCENT.

- [Math23k](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=Math23k>): Higher is better; unit: PERCENT.

- [Ape210k](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=Ape210k>): Higher is better; unit: PERCENT.

- [Gaokao Math](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=Gaokao%20Math>): Higher is better; unit: PERCENT.

- [AGIE Gaokao Math](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=AGIE%20Gaokao%20Math>): Higher is better; unit: PERCENT.

- [AGIE SAT Math](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=AGIE%20SAT%20Math>): Higher is better; unit: PERCENT.

- [Average Score](<https://www.sota2.com/research/sota/mathematical-reasoning-on-mwpbench-out-of-domain.md?metric=Average%20Score>): Higher is better; unit: PERCENT.

## Recorded results

| Method | College Math | Result date | Report paper |
| --- | --- | --- | --- |
| WizardMath Mistral | 24.8 | Not recorded | 68fa791f1e209131cea7c3cc |
| GPT-4 | 24.4 | Not recorded | 68fa791f1e209131cea7c3cc |
| WizardMath | 22.9 | Not recorded | 68fa791f1e209131cea7c3cc |
| MathScale Mistral | 21.8 | Not recorded | 68fa791f1e209131cea7c3cc |
| GPT-3.5-Turbo | 21.6 | Not recorded | 68fa791f1e209131cea7c3cc |
| WizardMath | 21.2 | Not recorded | 68fa791f1e209131cea7c3cc |
| MathScale 7B | 20.9 | Not recorded | 68fa791f1e209131cea7c3cc |
| MathScale 13B | 20.4 | Not recorded | 68fa791f1e209131cea7c3cc |
| MetaMath Mistral | 15.7 | Not recorded | 68fa791f1e209131cea7c3cc |
| MetaMath | 10.1 | Not recorded | 68fa791f1e209131cea7c3cc |
| MetaMath | 9.4 | Not recorded | 68fa791f1e209131cea7c3cc |
| GAIR-Abel | 7.9 | Not recorded | 68fa791f1e209131cea7c3cc |
| Mistral 7B | 7.5 | Not recorded | 68fa791f1e209131cea7c3cc |
| GAIR-Abel | 6.6 | Not recorded | 68fa791f1e209131cea7c3cc |
| MAmmoTH-CoT | 6.5 | Not recorded | 68fa791f1e209131cea7c3cc |
| MAmmoTH-CoT | 6.2 | Not recorded | 68fa791f1e209131cea7c3cc |
| LLaMA-2 7B | 2.3 | Not recorded | 68fa791f1e209131cea7c3cc |
| LLaMA-2 13B | 1.2 | Not recorded | 68fa791f1e209131cea7c3cc |

### WizardMath Mistral

College Math: 24.8. Result ID: 6995c7d23badfaffe6cb808c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Implementation](<https://github.com/nlpxucan/WizardLM>)

Recorded evaluation configuration: {"Backbone":"Mistral 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### GPT-4

College Math: 24.4. Result ID: 6995c7d23badfaffe6cb807b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Backbone":"Proprietary","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### WizardMath

College Math: 22.9. Result ID: 6995c7d23badfaffe6cb8082.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Implementation](<https://github.com/nlpxucan/WizardLM>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 13B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MathScale Mistral

College Math: 21.8. Result ID: 6995c7d23badfaffe6cb808b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72611e209131cea5611f.md>)

Recorded evaluation configuration: {"Backbone":"Mistral 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### GPT-3.5-Turbo

College Math: 21.6. Result ID: 6995c7d23badfaffe6cb807c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa727a1e209131cea56a1e.md>)

[Implementation](<https://github.com/openai/evals>)

Recorded evaluation configuration: {"Backbone":"Proprietary","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### WizardMath

College Math: 21.2. Result ID: 6995c7d23badfaffe6cb8088.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Implementation](<https://github.com/nlpxucan/WizardLM>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MathScale 7B

College Math: 20.9. Result ID: 6995c7d23badfaffe6cb8087.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72611e209131cea5611f.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MathScale 13B

College Math: 20.4. Result ID: 6995c7d23badfaffe6cb8081.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa72611e209131cea5611f.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 13B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MetaMath Mistral

College Math: 15.7. Result ID: 6995c7d23badfaffe6cb808a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73161e209131cea5a76d.md>)

[Implementation](<https://www.sota2.com/meta-math.github.io>)

Recorded evaluation configuration: {"Backbone":"Mistral 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MetaMath

College Math: 10.1. Result ID: 6995c7d23badfaffe6cb8080.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73161e209131cea5a76d.md>)

[Implementation](<https://www.sota2.com/meta-math.github.io>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 13B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MetaMath

College Math: 9.4. Result ID: 6995c7d23badfaffe6cb8086.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73161e209131cea5a76d.md>)

[Implementation](<https://www.sota2.com/meta-math.github.io>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### GAIR-Abel

College Math: 7.9. Result ID: 6995c7d23badfaffe6cb807f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 13B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### Mistral 7B

College Math: 7.5. Result ID: 6995c7d23badfaffe6cb8089.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70d61e209131cea4d909.md>)

[Implementation](<https://github.com/mistralai/mistral-src>)

Recorded evaluation configuration: {"Backbone":"Mistral 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### GAIR-Abel

College Math: 6.6. Result ID: 6995c7d23badfaffe6cb8085.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MAmmoTH-CoT

College Math: 6.5. Result ID: 6995c7d23badfaffe6cb807e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70b81e209131cea4cce0.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 13B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### MAmmoTH-CoT

College Math: 6.2. Result ID: 6995c7d23badfaffe6cb8084.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa70b81e209131cea4cce0.md>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### LLaMA-2 7B

College Math: 2.3. Result ID: 6995c7d23badfaffe6cb8083.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 7B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

### LLaMA-2 13B

College Math: 1.2. Result ID: 6995c7d23badfaffe6cb807d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa791f1e209131cea7c3cc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa701f1e209131cea492fe.md>)

[Implementation](<https://github.com/facebookresearch/llama>)

Recorded evaluation configuration: {"Backbone":"LLaMA-2 13B","Protocol":"CoT pass@1","Tool Usage":"No external python tool"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.