# Mathematical Reasoning on AIME 2024 (avg@64, pass@64)

Source: [Mathematical Reasoning on AIME 2024 (avg@64, pass@64)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-aime-2024-avg-at-64-pass-at-64>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-18T06:57:40.256Z

Evaluation of model performance on the American Invitational Mathematics Examination (AIME) 2024 dataset, reporting accuracy and pass rate metrics calculated using k=64 sampling.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | AIME 2024 |
| Metric | Average Score (avg@64) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Average Score (avg@64)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-aime-2024-avg-at-64-pass-at-64.md?metric=Average%20Score%20(avg%4064)>): Higher is better; unit: PERCENT.

- [Pass Rate (pass@64)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-aime-2024-avg-at-64-pass-at-64.md?metric=Pass%20Rate%20(pass%4064)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Average Score (avg@64) | Result date | Report paper |
| --- | --- | --- | --- |
| ASPO-Math-30B | 61.7 | Not recorded | 68fa7bf11e209131cea8b7bc |
| GRPO | 59.5 | Not recorded | 68fa7bf11e209131cea8b7bc |
| ASPO-Math-8B | 52.4 | Not recorded | 68fa7bf11e209131cea8b7bc |
| ASPO-Math-4B | 50.5 | Not recorded | 68fa7bf11e209131cea8b7bc |
| GRPO | 50.3 | Not recorded | 68fa7bf11e209131cea8b7bc |
| ASPO-Math-1.5B | 49 | Not recorded | 68fa7bf11e209131cea8b7bc |
| Nemotron-1.5B | 48 | Not recorded | 68fa7bf11e209131cea8b7bc |
| GRPO | 43.4 | Not recorded | 68fa7bf11e209131cea8b7bc |
| GRPO | 42.1 | Not recorded | 68fa7bf11e209131cea8b7bc |
| DeepScaleR-1.5B | 42 | Not recorded | 68fa7bf11e209131cea8b7bc |
| DeepSeek-R1-1.5B | 30.6 | Not recorded | 68fa7bf11e209131cea8b7bc |
| Qwen3-30B-A3B | 30.1 | Not recorded | 68fa7bf11e209131cea8b7bc |
| Qwen3-8B | 27 | Not recorded | 68fa7bf11e209131cea8b7bc |
| Qwen3-4B | 23.6 | Not recorded | 68fa7bf11e209131cea8b7bc |

### ASPO-Math-30B

Average Score (avg@64): 61.7. Result ID: 6a0ab74047f34ce924f0460d.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B"}. This record does not assert independent reproduction.

### GRPO

Average Score (avg@64): 59.5. Result ID: 6a0ab74047f34ce924f0460c.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B"}. This record does not assert independent reproduction.

### ASPO-Math-8B

Average Score (avg@64): 52.4. Result ID: 6a0ab74047f34ce924f0460a.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction.

### ASPO-Math-4B

Average Score (avg@64): 50.5. Result ID: 6a0ab74047f34ce924f04607.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction.

### GRPO

Average Score (avg@64): 50.3. Result ID: 6a0ab74047f34ce924f04609.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction.

### ASPO-Math-1.5B

Average Score (avg@64): 49. Result ID: 6a0ab74047f34ce924f04604.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction.

### Nemotron-1.5B

Average Score (avg@64): 48. Result ID: 6a0ab74047f34ce924f04603.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa79041e209131cea7bb2d.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction.

### GRPO

Average Score (avg@64): 43.4. Result ID: 6a0ab74047f34ce924f04606.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction.

### GRPO

Average Score (avg@64): 42.1. Result ID: 6a0ab74047f34ce924f04601.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa73101e209131cea5a395.md>)

[Implementation](<https://github.com/deepseek-ai/DeepSeek-Math>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction.

### DeepScaleR-1.5B

Average Score (avg@64): 42. Result ID: 6a0ab74047f34ce924f04602.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction.

### DeepSeek-R1-1.5B

Average Score (avg@64): 30.6. Result ID: 6a0ab74047f34ce924f04600.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

Recorded evaluation configuration: {"Backbone":"DeepSeek-R1-Distill-Qwen-1.5B"}. This record does not assert independent reproduction.

### Qwen3-30B-A3B

Average Score (avg@64): 30.1. Result ID: 6a0ab74047f34ce924f0460b.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-30B-A3B"}. This record does not assert independent reproduction.

### Qwen3-8B

Average Score (avg@64): 27. Result ID: 6a0ab74047f34ce924f04608.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-8B"}. This record does not assert independent reproduction.

### Qwen3-4B

Average Score (avg@64): 23.6. Result ID: 6a0ab74047f34ce924f04605.

[Reporting paper](<https://www.sota2.com/research/paper/68fa7bf11e209131cea8b7bc.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Backbone":"Qwen3-4B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
