# Mathematical Reasoning on MATH 500

Source: [Mathematical Reasoning on MATH 500](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-500>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-24T13:59:27.828Z

Evaluation of model performance on mathematical reasoning problems using the MATH 500 dataset, measured by pass@1.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | MATH 500 |
| Metric | pass@1 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 239 of 239 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [pass@1](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-500.md?metric=pass%401>): Higher is better; unit: PERCENT.

## Recorded results

| Method | pass@1 | Result date | Report paper |
| --- | --- | --- | --- |
| r1 | 97.3 | Not recorded | 69bcd48a0636e5bf4afc29e3 |
| OpenMath-Nemotron-14B + iGRPO | 96.9 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| OpenMath-Nemotron-14B + iGRPO | 96.7 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| Qwen3-8B | 96.2 | Not recorded | 698d5eccc97a0b3e3b5e3f72 |
| OpenMath-Nemotron-7B + iGRPO | 96 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| OpenMath-Nemotron-14B + GRPO | 96 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| OpenMath-Nemotron-7B + GRPO | 95.6 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| OpenMath-Nemotron-7B | 95.55 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| OpenMath-Nemotron-14B | 95.55 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| OpenMath-Nemotron-14B | 95.55 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| GRPO | 94.8 | Not recorded | 698d5eccc97a0b3e3b5e3f72 |
| RePO | 94.8 | Not recorded | 698d5eccc97a0b3e3b5e3f72 |
| o1 | 94.8 | Not recorded | 69bcd48a0636e5bf4afc29e3 |
| Sigma-MoE-Tiny | 94.6 | Not recorded | 6944dbed423ea42265729982 |
| DeepSeek-R1-Distill-Llama-70B | 94.5 | Not recorded | 68fa76a91e209131cea6e3eb |
| DeepSeek-R1-Distill-Qwen-32B | 94.3 | Not recorded | 68fa76a91e209131cea6e3eb |
| r1-distill | 94.3 | Not recorded | 69bcd48a0636e5bf4afc29e3 |
| DeepSeek-R1-Distill-Qwen-14B + iGRPO | 94 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| LUFFY | 94 | Not recorded | 698d5eccc97a0b3e3b5e3f72 |
| DeepSeek-R1-Distill-Qwen-14B | 93.9 | Not recorded | 68fa76a91e209131cea6e3eb |
| DeepSeek-R1-Distill-Qwen-7B + iGRPO | 93.8 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| DeepSeek-R1-Distill-Qwen-7B + Self-Verification | 93.5 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| DeepSeek-R1-Distill-Qwen-7B + Critique-GRPO | 93.45 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| Qwen3-1.7B | 93.4 | Not recorded | 6944dbed423ea42265729982 |
| DeepSeek-R1-Distill-Qwen-7B + GRPO | 93.25 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| PCL | 93.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| Dynamic Sampling (Oracle) | 93.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| GPS | 93.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| DeepSeek-R1-Distill-Qwen-14B | 93.1 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| DeepSeek-R1-Distill-Qwen-14B + GRPO | 93.1 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| Bespoke-32B | 93 | Not recorded | 69bcd48a0636e5bf4afc29e3 |
| DeepSeek-R1-Distill-Qwen-7B | 92.8 | Not recorded | 68fa76a91e209131cea6e3eb |
| DeepSeek-R1-Distill-Qwen-7B | 92.8 | Not recorded | 698abc11c97a0b3e3b5de3a4 |
| DeepSeek-R1-Distill-Qwen-7B | 92.8 | Not recorded | 6944dbed423ea42265729982 |
| s1-32B | 92.6 | Not recorded | 69bcd48a0636e5bf4afc29e3 |
| MoPPS | 92 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| THINKSAFE | 91.9 | Not recorded | 69802fd8a8b01e76cc8e180d |
| GRESO | 91.8 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| Uniform Sampling | 91.6 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| SafeChain | 91.5 | Not recorded | 69802fd8a8b01e76cc8e180d |
| SafePath | 90.62 | Not recorded | 69802fd8a8b01e76cc8e180d |
| HEAL | 90.62 | Not recorded | 69b24879d030084c22e2f864 |
| QwQ-32B | 90.6 | Not recorded | 69bcd48a0636e5bf4afc29e3 |
| STAR-1 | 90.58 | Not recorded | 69802fd8a8b01e76cc8e180d |
| MulFeRL | 90.24 | Not recorded | 69802f81a8b01e76cc8e15d7 |
| Initial | 90.18 | Not recorded | 69802fd8a8b01e76cc8e180d |
| o1-mini | 90 | Not recorded | 69bcd48a0636e5bf4afc29e3 |
| SafeKey | 89.9 | Not recorded | 69802fd8a8b01e76cc8e180d |
| DirectRefusal | 89.82 | Not recorded | 69802fd8a8b01e76cc8e180d |
| DeepSeek-R1-Distill-Llama-8B | 89.1 | Not recorded | 68fa76a91e209131cea6e3eb |
| DeepSeek-R1-Distill-Llama-8B | 89.1 | Not recorded | 6944dbed423ea42265729982 |
| Critique-GRPO | 88.64 | Not recorded | 69802f81a8b01e76cc8e15d7 |
| Dr.GRPO | 88.28 | Not recorded | 69802f81a8b01e76cc8e15d7 |
| GPS | 88 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| GRPO | 87.84 | Not recorded | 69802f81a8b01e76cc8e15d7 |
| Curriculum SFT | 87.82 | Not recorded | 69b24879d030084c22e2f864 |
| THINKSAFE | 87.7 | Not recorded | 69802fd8a8b01e76cc8e180d |
| SafePath | 87.43 | Not recorded | 69802fd8a8b01e76cc8e180d |
| Initial | 87.38 | Not recorded | 69802fd8a8b01e76cc8e180d |
| Rule+CER | 87.3 | Not recorded | 69b248afd030084c22e2fa78 |
| Dynamic Sampling (Oracle) | 87.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| DeepSeek-R1-Distill-7B | 87.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| CER | 87.2 | Not recorded | 69b248afd030084c22e2fa78 |
| CER | 87.2 | Not recorded | 69b248afd030084c22e2fa78 |
| HEAL | 87.18 | Not recorded | 69b24879d030084c22e2f864 |
| SFT | 87.06 | Not recorded | 69b24879d030084c22e2f864 |
| GEM | 87 | Not recorded | 698abb76c97a0b3e3b5ddeb8 |
| Rule-based | 86.7 | Not recorded | 69b248afd030084c22e2fa78 |
| STAR-1 | 86.65 | Not recorded | 69802fd8a8b01e76cc8e180d |
| SED-SFT | 86.6 | Not recorded | 698abb76c97a0b3e3b5ddeb8 |
| SafeChain | 86.5 | Not recorded | 69802fd8a8b01e76cc8e180d |
| Uniform Sampling | 86.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| GRESO | 86.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| MoPPS | 86.2 | Not recorded | 6981819ca8b01e76cc8e26e0 |
| SCRL | 86.2 | Not recorded | 69c0c8e70636e5bf4afc3a3f |
| SED-SFT w/o mask | 86 | Not recorded | 698abb76c97a0b3e3b5ddeb8 |
| Rule-based | 86 | Not recorded | 69b248afd030084c22e2fa78 |
| VeriFree | 86 | Not recorded | 69b248afd030084c22e2fa78 |
| General-verifier | 86 | Not recorded | 69b248afd030084c22e2fa78 |
| SafeKey | 85.8 | Not recorded | 69802fd8a8b01e76cc8e180d |

### r1

pass@1: 97.3. Result ID: 69bd01f047f34ce924a6f2ac.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Availability":"Open Weights","Reasoning finetuning examples (# ex.)":"≫800K"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-14B + iGRPO

pass@1: 96.9. Result ID: 6995f7bb5698665ec42be1c3.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

Recorded evaluation configuration: {"Backbone":"Nemotron-14B","Fine-tuning":"iGRPO"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-14B + iGRPO

pass@1: 96.7. Result ID: 6995f7bb5698665ec42be197.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

Recorded evaluation configuration: {"Base Model":"OpenMath-Nemotron-14B","Training Method":"iGRPO","Parameter Size":"14B"}. This record does not assert independent reproduction.

### Qwen3-8B

pass@1: 96.2. Result ID: 6998c4275698665ec4276484.

[Reporting paper](<https://www.sota2.com/research/paper/698d5eccc97a0b3e3b5e3f72.md>)

Recorded evaluation configuration: {"training\_data":"SuperGPQA subset (11k)"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-7B + iGRPO

pass@1: 96. Result ID: 6995f7bb5698665ec42be191.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

Recorded evaluation configuration: {"Base Model":"OpenMath-Nemotron-7B","Training Method":"iGRPO","Parameter Size":"7B"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-14B + GRPO

pass@1: 96. Result ID: 6995f7bb5698665ec42be196.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Base Model":"OpenMath-Nemotron-14B","Training Method":"GRPO","Parameter Size":"14B"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-7B + GRPO

pass@1: 95.6. Result ID: 6995f7bb5698665ec42be190.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Base Model":"OpenMath-Nemotron-7B","Training Method":"GRPO","Parameter Size":"7B"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-7B

pass@1: 95.55. Result ID: 6995f7bb5698665ec42be18f.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa782e1e209131cea77079.md>)

Recorded evaluation configuration: {"Base Model":"OpenMath-Nemotron-7B","Parameter Size":"7B"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-14B

pass@1: 95.55. Result ID: 6995f7bb5698665ec42be195.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa782e1e209131cea77079.md>)

Recorded evaluation configuration: {"Base Model":"OpenMath-Nemotron-14B","Parameter Size":"14B"}. This record does not assert independent reproduction.

### OpenMath-Nemotron-14B

pass@1: 95.55. Result ID: 6995f7bb5698665ec42be1c2.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa782e1e209131cea77079.md>)

Recorded evaluation configuration: {"Backbone":"Nemotron-14B"}. This record does not assert independent reproduction.

### GRPO

pass@1: 94.8. Result ID: 6998c4275698665ec4276485.

[Reporting paper](<https://www.sota2.com/research/paper/698d5eccc97a0b3e3b5e3f72.md>)

Recorded evaluation configuration: {"training\_data":"SuperGPQA subset (11k)"}. This record does not assert independent reproduction.

### RePO

pass@1: 94.8. Result ID: 6998c4275698665ec4276487.

[Reporting paper](<https://www.sota2.com/research/paper/698d5eccc97a0b3e3b5e3f72.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698d5eccc97a0b3e3b5e3f72.md>)

Recorded evaluation configuration: {"training\_data":"SuperGPQA subset (11k)"}. This record does not assert independent reproduction.

### o1

pass@1: 94.8. Result ID: 69bd01f047f34ce924a6f2a9.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

Recorded evaluation configuration: {"Availability":"API only","Reasoning finetuning examples (# ex.)":"N.A."}. This record does not assert independent reproduction.

### Sigma-MoE-Tiny

pass@1: 94.6. Result ID: 6996baae5698665ec4056d47.

[Reporting paper](<https://www.sota2.com/research/paper/6944dbed423ea42265729982.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6944dbed423ea42265729982.md>)

Recorded evaluation configuration: {"Architecture":"MoE","# Activated Params":"0.5B","# Total Params":"20B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Llama-70B

pass@1: 94.5. Result ID: 69945c4a3a6c3148491cbe52.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Architecture":"Llama","Parameter count":"70B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-32B

pass@1: 94.3. Result ID: 69945c4a3a6c3148491cbe50.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Architecture":"Qwen","Parameter count":"32B"}. This record does not assert independent reproduction.

### r1-distill

pass@1: 94.3. Result ID: 69bd01f047f34ce924a6f2ad.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Availability":"Open Weights","Reasoning finetuning examples (# ex.)":"800K"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-14B + iGRPO

pass@1: 94. Result ID: 6995f7bb5698665ec42be194.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-14B","Training Method":"iGRPO","Parameter Size":"14B"}. This record does not assert independent reproduction.

### LUFFY

pass@1: 94. Result ID: 6998c4275698665ec4276486.

[Reporting paper](<https://www.sota2.com/research/paper/698d5eccc97a0b3e3b5e3f72.md>)

Recorded evaluation configuration: {"training\_data":"SuperGPQA subset (11k)"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-14B

pass@1: 93.9. Result ID: 69945c4a3a6c3148491cbe4f.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Architecture":"Qwen","Parameter count":"14B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-7B + iGRPO

pass@1: 93.8. Result ID: 6995f7bb5698665ec42be18e.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-7B","Training Method":"iGRPO","Parameter Size":"7B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-7B + Self-Verification

pass@1: 93.5. Result ID: 6995f7bb5698665ec42be18c.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79001e209131cea7b999.md>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-7B","Training Method":"Self-Verification","Parameter Size":"7B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-7B + Critique-GRPO

pass@1: 93.45. Result ID: 6995f7bb5698665ec42be18d.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ac21e209131cea850cf.md>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-7B","Training Method":"Critique-GRPO","Parameter Size":"7B"}. This record does not assert independent reproduction.

### Qwen3-1.7B

pass@1: 93.4. Result ID: 6996baae5698665ec4056d45.

[Reporting paper](<https://www.sota2.com/research/paper/6944dbed423ea42265729982.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Architecture":"Dense","# Activated Params":"1.7B","# Total Params":"1.7B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-7B + GRPO

pass@1: 93.25. Result ID: 6995f7bb5698665ec42be18b.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-7B","Training Method":"GRPO","Parameter Size":"7B"}. This record does not assert independent reproduction.

### PCL

pass@1: 93.2. Result ID: 69960c525698665ec446729f.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bc51e209131cea8a585.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-7B","Strategy":"PCL","Runtime":"50h"}. This record does not assert independent reproduction.

### Dynamic Sampling (Oracle)

pass@1: 93.2. Result ID: 69960c525698665ec44672a2.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea73a27.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-7B","Strategy":"DS","Runtime":"77h"}. This record does not assert independent reproduction.

### GPS

pass@1: 93.2. Result ID: 69960c525698665ec44672a3.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-7B","Strategy":"GPS","Runtime":"49h"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-14B

pass@1: 93.1. Result ID: 6995f7bb5698665ec42be192.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-14B","Parameter Size":"14B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-14B + GRPO

pass@1: 93.1. Result ID: 6995f7bb5698665ec42be193.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-14B","Training Method":"GRPO","Parameter Size":"14B"}. This record does not assert independent reproduction.

### Bespoke-32B

pass@1: 93. Result ID: 69bd01f047f34ce924a6f2af.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

Recorded evaluation configuration: {"Availability":"Open Weights and Open Data","Reasoning finetuning examples (# ex.)":"17K"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-7B

pass@1: 92.8. Result ID: 69945c4a3a6c3148491cbe4e.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Architecture":"Qwen","Parameter count":"7B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-7B

pass@1: 92.8. Result ID: 6995f7bb5698665ec42be18a.

[Reporting paper](<https://www.sota2.com/research/paper/698abc11c97a0b3e3b5de3a4.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Base Model":"DeepSeek-R1-Distill-Qwen-7B","Parameter Size":"7B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Qwen-7B

pass@1: 92.8. Result ID: 6996baae5698665ec4056d43.

[Reporting paper](<https://www.sota2.com/research/paper/6944dbed423ea42265729982.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Architecture":"Dense","# Activated Params":"7B","# Total Params":"7B"}. This record does not assert independent reproduction.

### s1-32B

pass@1: 92.6. Result ID: 69bd01f047f34ce924a6f2b0.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

Recorded evaluation configuration: {"Availability":"Open Weights and Open Data","Reasoning finetuning examples (# ex.)":"1K"}. This record does not assert independent reproduction.

### MoPPS

pass@1: 92. Result ID: 69960c525698665ec44672a1.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c0f1e209131cea8c2df.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-7B","Strategy":"MoPPS","Runtime":"42h"}. This record does not assert independent reproduction.

### THINKSAFE

pass@1: 91.9. Result ID: 699614075698665ec45379e1.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Implementation](<https://github.com/seanie12/ThinkSafe.git>)

Recorded evaluation configuration: {"Model Size":"7B","Sampling trajectories":8}. This record does not assert independent reproduction.

### GRESO

pass@1: 91.8. Result ID: 69960c525698665ec44672a0.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79181e209131cea7c1cb.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-7B","Strategy":"GRESO","Runtime":"53h"}. This record does not assert independent reproduction.

### Uniform Sampling

pass@1: 91.6. Result ID: 69960c525698665ec446729e.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-7B","Strategy":"Uniform","Runtime":"40h"}. This record does not assert independent reproduction.

### SafeChain

pass@1: 91.5. Result ID: 699614075698665ec45379dd.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76ff1e209131cea7058f.md>)

Recorded evaluation configuration: {"Model Size":"7B","Sampling trajectories":8}. This record does not assert independent reproduction.

### SafePath

pass@1: 90.62. Result ID: 699614075698665ec45379df.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790c1e209131cea7bde6.md>)

Recorded evaluation configuration: {"Model Size":"7B","Sampling trajectories":8}. This record does not assert independent reproduction.

### HEAL

pass@1: 90.62. Result ID: 69b2707e6dbb7075b95ccb49.

[Reporting paper](<https://www.sota2.com/research/paper/69b24879d030084c22e2f864.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b24879d030084c22e2f864.md>)

Recorded evaluation configuration: {"Models":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction.

### QwQ-32B

pass@1: 90.6. Result ID: 69bd01f047f34ce924a6f2ab.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa767e1e209131cea6d6a7.md>)

Recorded evaluation configuration: {"Availability":"Open Weights","Reasoning finetuning examples (# ex.)":"N.A."}. This record does not assert independent reproduction.

### STAR-1

pass@1: 90.58. Result ID: 699614075698665ec45379de.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77c81e209131cea754b3.md>)

Recorded evaluation configuration: {"Model Size":"7B","Sampling trajectories":8}. This record does not assert independent reproduction.

### MulFeRL

pass@1: 90.24. Result ID: 6996149f5698665ec45421cb.

[Reporting paper](<https://www.sota2.com/research/paper/69802f81a8b01e76cc8e15d7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802f81a8b01e76cc8e15d7.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Inst","Training Strategy":"Multi-turn Feedback-guided Reinforcement Learning"}. This record does not assert independent reproduction.

### Initial

pass@1: 90.18. Result ID: 699614075698665ec45379db.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

Recorded evaluation configuration: {"Model Size":"7B","Sampling trajectories":8}. This record does not assert independent reproduction.

### o1-mini

pass@1: 90. Result ID: 69bd01f047f34ce924a6f2a8.

[Reporting paper](<https://www.sota2.com/research/paper/69bcd48a0636e5bf4afc29e3.md>)

Recorded evaluation configuration: {"Availability":"API only","Reasoning finetuning examples (# ex.)":"N.A."}. This record does not assert independent reproduction.

### SafeKey

pass@1: 89.9. Result ID: 699614075698665ec45379e0.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78be1e209131cea7a311.md>)

Recorded evaluation configuration: {"Model Size":"7B","Sampling trajectories":8}. This record does not assert independent reproduction.

### DirectRefusal

pass@1: 89.82. Result ID: 699614075698665ec45379dc.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79321e209131cea7c8ee.md>)

Recorded evaluation configuration: {"Model Size":"7B","Sampling trajectories":8}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Llama-8B

pass@1: 89.1. Result ID: 69945c4a3a6c3148491cbe51.

[Reporting paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Architecture":"Llama","Parameter count":"8B"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-Llama-8B

pass@1: 89.1. Result ID: 6996baae5698665ec4056d44.

[Reporting paper](<https://www.sota2.com/research/paper/6944dbed423ea42265729982.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Architecture":"Dense","# Activated Params":"8B","# Total Params":"8B"}. This record does not assert independent reproduction.

### Critique-GRPO

pass@1: 88.64. Result ID: 6996149f5698665ec45421ca.

[Reporting paper](<https://www.sota2.com/research/paper/69802f81a8b01e76cc8e15d7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7ac21e209131cea850cf.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Inst","Training Strategy":"Reinforcement Learning-based Finetuning"}. This record does not assert independent reproduction.

### Dr.GRPO

pass@1: 88.28. Result ID: 6996149f5698665ec45421c9.

[Reporting paper](<https://www.sota2.com/research/paper/69802f81a8b01e76cc8e15d7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7be51e209131cea8b201.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Inst","Training Strategy":"Reinforcement Learning-based Finetuning"}. This record does not assert independent reproduction.

### GPS

pass@1: 88. Result ID: 69960c525698665ec446729c.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-1.5B","Strategy":"GPS","Runtime":"16h"}. This record does not assert independent reproduction.

### GRPO

pass@1: 87.84. Result ID: 6996149f5698665ec45421c8.

[Reporting paper](<https://www.sota2.com/research/paper/69802f81a8b01e76cc8e15d7.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76a91e209131cea6e3eb.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B-Inst","Training Strategy":"Reinforcement Learning-based Finetuning"}. This record does not assert independent reproduction.

### Curriculum SFT

pass@1: 87.82. Result ID: 69b2707e6dbb7075b95ccb48.

[Reporting paper](<https://www.sota2.com/research/paper/69b24879d030084c22e2f864.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78d41e209131cea7aa25.md>)

Recorded evaluation configuration: {"Models":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction.

### THINKSAFE

pass@1: 87.7. Result ID: 699614085698665ec45379e8.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Implementation](<https://github.com/seanie12/ThinkSafe.git>)

Recorded evaluation configuration: {"Model Size":"8B","Sampling trajectories":8}. This record does not assert independent reproduction.

### SafePath

pass@1: 87.43. Result ID: 699614075698665ec45379e6.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790c1e209131cea7bde6.md>)

Recorded evaluation configuration: {"Model Size":"8B","Sampling trajectories":8}. This record does not assert independent reproduction.

### Initial

pass@1: 87.38. Result ID: 699614075698665ec45379e2.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

Recorded evaluation configuration: {"Model Size":"8B","Sampling trajectories":8}. This record does not assert independent reproduction.

### Rule+CER

pass@1: 87.3. Result ID: 69b27f656dbb7075b95d3f4c.

[Reporting paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction.

### Dynamic Sampling (Oracle)

pass@1: 87.2. Result ID: 69960c525698665ec446729b.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77801e209131cea73a27.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-1.5B","Strategy":"DS","Runtime":"30h"}. This record does not assert independent reproduction.

### DeepSeek-R1-Distill-7B

pass@1: 87.2. Result ID: 69960c525698665ec446729d.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-7B","Strategy":"Base"}. This record does not assert independent reproduction.

### CER

pass@1: 87.2. Result ID: 69b27f656dbb7075b95d3f05.

[Reporting paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Reward Method":"CER","Training Data Domain":"General-domain"}. This record does not assert independent reproduction.

### CER

pass@1: 87.2. Result ID: 69b27f656dbb7075b95d3f4b.

[Reporting paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction.

### HEAL

pass@1: 87.18. Result ID: 69b2707e6dbb7075b95ccb4e.

[Reporting paper](<https://www.sota2.com/research/paper/69b24879d030084c22e2f864.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69b24879d030084c22e2f864.md>)

Recorded evaluation configuration: {"Models":"Qwen3-4B-Base"}. This record does not assert independent reproduction.

### SFT

pass@1: 87.06. Result ID: 69b2707e6dbb7075b95ccb47.

[Reporting paper](<https://www.sota2.com/research/paper/69b24879d030084c22e2f864.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f701e209131cea45952.md>)

Recorded evaluation configuration: {"Models":"Qwen2.5-14B-Instruct"}. This record does not assert independent reproduction.

### GEM

pass@1: 87. Result ID: 699642e65698665ec486802b.

[Reporting paper](<https://www.sota2.com/research/paper/698abb76c97a0b3e3b5ddeb8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77dd1e209131cea75a95.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B-Instruct","Training Phase":"RL","Optimization Objective":"GEM"}. This record does not assert independent reproduction.

### Rule-based

pass@1: 86.7. Result ID: 69b27f656dbb7075b95d3f48.

[Reporting paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction.

### STAR-1

pass@1: 86.65. Result ID: 699614075698665ec45379e5.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa77c81e209131cea754b3.md>)

Recorded evaluation configuration: {"Model Size":"8B","Sampling trajectories":8}. This record does not assert independent reproduction.

### SED-SFT

pass@1: 86.6. Result ID: 699642e65698665ec486802e.

[Reporting paper](<https://www.sota2.com/research/paper/698abb76c97a0b3e3b5ddeb8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abb76c97a0b3e3b5ddeb8.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B-Instruct","Training Phase":"RL","Optimization Objective":"SED-SFT"}. This record does not assert independent reproduction.

### SafeChain

pass@1: 86.5. Result ID: 699614075698665ec45379e4.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa76ff1e209131cea7058f.md>)

Recorded evaluation configuration: {"Model Size":"8B","Sampling trajectories":8}. This record does not assert independent reproduction.

### Uniform Sampling

pass@1: 86.2. Result ID: 69960c525698665ec4467297.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-1.5B","Strategy":"Uniform","Runtime":"16h"}. This record does not assert independent reproduction.

### GRESO

pass@1: 86.2. Result ID: 69960c525698665ec4467299.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79181e209131cea7c1cb.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-1.5B","Strategy":"GRESO","Runtime":"27h"}. This record does not assert independent reproduction.

### MoPPS

pass@1: 86.2. Result ID: 69960c525698665ec446729a.

[Reporting paper](<https://www.sota2.com/research/paper/6981819ca8b01e76cc8e26e0.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7c0f1e209131cea8c2df.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-1.5B","Strategy":"MoPPS","Runtime":"17h"}. This record does not assert independent reproduction.

### SCRL

pass@1: 86.2. Result ID: 69c124c447f34ce924a9a2fb.

[Reporting paper](<https://www.sota2.com/research/paper/69c0c8e70636e5bf4afc3a3f.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69c0c8e70636e5bf4afc3a3f.md>)

Recorded evaluation configuration: {"Candidate responses":"64","Training samples":"32","Backbone":"Qwen2.5-Math-7B"}. This record does not assert independent reproduction.

### SED-SFT w/o mask

pass@1: 86. Result ID: 699642e65698665ec486802d.

[Reporting paper](<https://www.sota2.com/research/paper/698abb76c97a0b3e3b5ddeb8.md>)

[Method origin paper](<https://www.sota2.com/research/paper/698abb76c97a0b3e3b5ddeb8.md>)

Recorded evaluation configuration: {"Backbone":"Qwen2.5-Math-7B-Instruct","Training Phase":"RL","Optimization Objective":"SED-SFT w/o mask"}. This record does not assert independent reproduction.

### Rule-based

pass@1: 86. Result ID: 69b27f656dbb7075b95d3f02.

[Reporting paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Reward Method":"Rule-based","Training Data Domain":"General-domain"}. This record does not assert independent reproduction.

### VeriFree

pass@1: 86. Result ID: 69b27f656dbb7075b95d3f03.

[Reporting paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78d11e209131cea7a946.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Reward Method":"VeriFree","Training Data Domain":"General-domain"}. This record does not assert independent reproduction.

### General-verifier

pass@1: 86. Result ID: 69b27f656dbb7075b95d3f4a.

[Reporting paper](<https://www.sota2.com/research/paper/69b248afd030084c22e2fa78.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa793e1e209131cea7cd45.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B-Base","Training Dataset":"mathematical dataset"}. This record does not assert independent reproduction.

### SafeKey

pass@1: 85.8. Result ID: 699614075698665ec45379e7.

[Reporting paper](<https://www.sota2.com/research/paper/69802fd8a8b01e76cc8e180d.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa78be1e209131cea7a311.md>)

Recorded evaluation configuration: {"Model Size":"8B","Sampling trajectories":8}. This record does not assert independent reproduction.

Page 1 of 3. This is an index page, not the full collection.

- [Next index page](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-500.md?metric=pass%401&page=2>): Follow this link to discover more pages.
