# Mathematical Reasoning on MATH500 (Pass@1, FS, N/T, D/T, Repro)

Source: [Mathematical Reasoning on MATH500 (Pass@1, FS, N/T, D/T, Repro)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math500-pass-at-1-fs-n-t-d-t-repro>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-06T08:41:43.472Z

Evaluation of model performance on the MATH500 dataset for mathematical reasoning, reporting key metrics including Pass@1, Few-Shot (FS), Normalized/Total (N/T), Direct/Total (D/T), and Reproducibility (Repro).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | MATH500 |
| Metric | Pass@1 Accuracy |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Pass@1 Accuracy](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math500-pass-at-1-fs-n-t-d-t-repro.md?metric=Pass%401%20Accuracy>): Higher is better; unit: PERCENT.

- [Few-Shot Score (FS)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math500-pass-at-1-fs-n-t-d-t-repro.md?metric=Few-Shot%20Score%20(FS)>): Higher is better; unit: SCALAR.

- [Normalized/Total Ratio (N/T)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math500-pass-at-1-fs-n-t-d-t-repro.md?metric=Normalized%2FTotal%20Ratio%20(N%2FT)>): Higher is better; unit: SCALAR.

- [Direct/Total Ratio (D/T)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math500-pass-at-1-fs-n-t-d-t-repro.md?metric=Direct%2FTotal%20Ratio%20(D%2FT)>): Higher is better; unit: SCALAR.

- [Reproducibility Rate (Repro)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math500-pass-at-1-fs-n-t-d-t-repro.md?metric=Reproducibility%20Rate%20(Repro)>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Pass@1 Accuracy | Result date | Report paper |
| --- | --- | --- | --- |
| RED | 97.9 | Not recorded | 69d33dea7d0172140ec735b1 |
| S-GRRO | 97.3 | Not recorded | 69d33dea7d0172140ec735b1 |
| RL + LP | 97.2 | Not recorded | 69d33dea7d0172140ec735b1 |
| Vanilla | 96.8 | Not recorded | 69d33dea7d0172140ec735b1 |
| DEER | 96.4 | Not recorded | 69d33dea7d0172140ec735b1 |
| RED | 96.2 | Not recorded | 69d33dea7d0172140ec735b1 |
| RED | 95.2 | Not recorded | 69d33dea7d0172140ec735b1 |
| S-GRRO | 95.1 | Not recorded | 69d33dea7d0172140ec735b1 |
| S-GRRO | 94.2 | Not recorded | 69d33dea7d0172140ec735b1 |
| Vanilla | 94.1 | Not recorded | 69d33dea7d0172140ec735b1 |
| RL + LP | 93.8 | Not recorded | 69d33dea7d0172140ec735b1 |
| Vanilla | 93.3 | Not recorded | 69d33dea7d0172140ec735b1 |
| RL + LP | 93.1 | Not recorded | 69d33dea7d0172140ec735b1 |
| DEER | 92.3 | Not recorded | 69d33dea7d0172140ec735b1 |
| DEER | 91.8 | Not recorded | 69d33dea7d0172140ec735b1 |

### RED

Pass@1 Accuracy: 97.9. Result ID: 69d371c547f34ce924bb5948.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-32B-Thinking"}. This record does not assert independent reproduction.

### S-GRRO

Pass@1 Accuracy: 97.3. Result ID: 69d371c547f34ce924bb5947.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78961e209131cea79249.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-32B-Thinking"}. This record does not assert independent reproduction.

### RL + LP

Pass@1 Accuracy: 97.2. Result ID: 69d371c547f34ce924bb5946.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a81e209131cea79a9b.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-32B-Thinking"}. This record does not assert independent reproduction.

### Vanilla

Pass@1 Accuracy: 96.8. Result ID: 69d371c547f34ce924bb5944.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-32B-Thinking"}. This record does not assert independent reproduction.

### DEER

Pass@1 Accuracy: 96.4. Result ID: 69d371c547f34ce924bb5945.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ba11e209131cea89778.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-32B-Thinking"}. This record does not assert independent reproduction.

### RED

Pass@1 Accuracy: 96.2. Result ID: 69d371c547f34ce924bb5952.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

Recorded evaluation configuration: {"Backbone":"R1-Llama-70B"}. This record does not assert independent reproduction.

### RED

Pass@1 Accuracy: 95.2. Result ID: 69d371c547f34ce924bb594d.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

Recorded evaluation configuration: {"Backbone":"R1-Qwen-32B"}. This record does not assert independent reproduction.

### S-GRRO

Pass@1 Accuracy: 95.1. Result ID: 69d371c547f34ce924bb5951.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78961e209131cea79249.md>)

Recorded evaluation configuration: {"Backbone":"R1-Llama-70B"}. This record does not assert independent reproduction.

### S-GRRO

Pass@1 Accuracy: 94.2. Result ID: 69d371c547f34ce924bb594c.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78961e209131cea79249.md>)

Recorded evaluation configuration: {"Backbone":"R1-Qwen-32B"}. This record does not assert independent reproduction.

### Vanilla

Pass@1 Accuracy: 94.1. Result ID: 69d371c547f34ce924bb594e.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

Recorded evaluation configuration: {"Backbone":"R1-Llama-70B"}. This record does not assert independent reproduction.

### RL + LP

Pass@1 Accuracy: 93.8. Result ID: 69d371c547f34ce924bb5950.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a81e209131cea79a9b.md>)

Recorded evaluation configuration: {"Backbone":"R1-Llama-70B"}. This record does not assert independent reproduction.

### Vanilla

Pass@1 Accuracy: 93.3. Result ID: 69d371c547f34ce924bb5949.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

Recorded evaluation configuration: {"Backbone":"R1-Qwen-32B"}. This record does not assert independent reproduction.

### RL + LP

Pass@1 Accuracy: 93.1. Result ID: 69d371c547f34ce924bb594b.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78a81e209131cea79a9b.md>)

Recorded evaluation configuration: {"Backbone":"R1-Qwen-32B"}. This record does not assert independent reproduction.

### DEER

Pass@1 Accuracy: 92.3. Result ID: 69d371c547f34ce924bb594f.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ba11e209131cea89778.md>)

Recorded evaluation configuration: {"Backbone":"R1-Llama-70B"}. This record does not assert independent reproduction.

### DEER

Pass@1 Accuracy: 91.8. Result ID: 69d371c547f34ce924bb594a.

[Reporting paper](<https://www.sota2.com/research/paper/69d33dea7d0172140ec735b1.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ba11e209131cea89778.md>)

Recorded evaluation configuration: {"Backbone":"R1-Qwen-32B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
