# Mathematical Reasoning on SVAMP (Pass@5, # Tokens)

Source: [Mathematical Reasoning on SVAMP (Pass@5, # Tokens)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-svamp-pass-at-5-sharp-tokens>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:57:36.269Z

Evaluation of model performance on mathematical reasoning tasks using the SVAMP dataset, specifically measured using Pass@5 and token count.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | SVAMP |
| Metric | Pass@5 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Pass@5](<https://www.sota2.com/research/sota/mathematical-reasoning-on-svamp-pass-at-5-sharp-tokens.md?metric=Pass%405>): Higher is better; unit: PERCENT.

- [Token Count](<https://www.sota2.com/research/sota/mathematical-reasoning-on-svamp-pass-at-5-sharp-tokens.md?metric=Token%20Count>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Pass@5 | Result date | Report paper |
| --- | --- | --- | --- |
| 0-CoT | 97 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 97 | Not recorded | 692d210bd6b0dd3342931651 |
| 0-CoT | 97 | Not recorded | 692d210bd6b0dd3342931651 |
| PS | 97 | Not recorded | 692d210bd6b0dd3342931651 |
| 0-CoT | 97 | Not recorded | 692d210bd6b0dd3342931651 |
| PS | 96.67 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 96.67 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 96.33 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 96.33 | Not recorded | 692d210bd6b0dd3342931651 |
| PS | 96 | Not recorded | 692d210bd6b0dd3342931651 |
| PS | 96 | Not recorded | 692d210bd6b0dd3342931651 |
| 0-CoT | 95.67 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 95.67 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 95.67 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 95 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 94.33 | Not recorded | 692d210bd6b0dd3342931651 |

### 0-CoT

Pass@5: 97. Result ID: 6995de563badfaffe6e5548d.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 97. Result ID: 6995de563badfaffe6e5548f.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### 0-CoT

Pass@5: 97. Result ID: 6995de563badfaffe6e55491.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### PS

Pass@5: 97. Result ID: 6995de563badfaffe6e55492.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### 0-CoT

Pass@5: 97. Result ID: 6995de563badfaffe6e55495.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### PS

Pass@5: 96.67. Result ID: 6995de563badfaffe6e5548e.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 96.67. Result ID: 6995de563badfaffe6e55493.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 96.33. Result ID: 6995de563badfaffe6e5548b.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 96.33. Result ID: 6995de563badfaffe6e55497.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### PS

Pass@5: 96. Result ID: 6995de563badfaffe6e5548a.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

### PS

Pass@5: 96. Result ID: 6995de563badfaffe6e55496.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### 0-CoT

Pass@5: 95.67. Result ID: 6995de563badfaffe6e55489.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 95.67. Result ID: 6995de563badfaffe6e55494.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 95.67. Result ID: 6995de563badfaffe6e55498.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 95. Result ID: 6995de563badfaffe6e55490.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 94.33. Result ID: 6995de563badfaffe6e5548c.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
