# Mathematical Reasoning on AIME 2024 (Pass@5)

Source: [Mathematical Reasoning on AIME 2024 (Pass@5)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-aime-2024-pass-at-5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-26T05:57:36.287Z

Evaluation of model performance on the American Invitational Mathematics Examination (AIME) 2024 dataset, specifically measured using the Pass@5 metric and reporting the total number of generated tokens.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Mathematical Reasoning |
| Dataset (including any stated split) | AIME 2024 |
| Metric | Pass@5 |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 16 of 16 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Pass@5](<https://www.sota2.com/research/sota/mathematical-reasoning-on-aime-2024-pass-at-5.md?metric=Pass%405>): Higher is better; unit: PERCENT.

- [Total Tokens](<https://www.sota2.com/research/sota/mathematical-reasoning-on-aime-2024-pass-at-5.md?metric=Total%20Tokens>): Lower is better; unit: SCALAR.

## Recorded results

| Method | Pass@5 | Result date | Report paper |
| --- | --- | --- | --- |
| PS | 90 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 90 | Not recorded | 692d210bd6b0dd3342931651 |
| 0-CoT | 90 | Not recorded | 692d210bd6b0dd3342931651 |
| PS | 90 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 90 | Not recorded | 692d210bd6b0dd3342931651 |
| 0-CoT | 86.67 | Not recorded | 692d210bd6b0dd3342931651 |
| 0-CoT | 86.67 | Not recorded | 692d210bd6b0dd3342931651 |
| PS | 83.33 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 83.33 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 83.33 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 83.33 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 83.33 | Not recorded | 692d210bd6b0dd3342931651 |
| 0-CoT | 30 | Not recorded | 692d210bd6b0dd3342931651 |
| PS | 30 | Not recorded | 692d210bd6b0dd3342931651 |
| F-CoT | 26.67 | Not recorded | 692d210bd6b0dd3342931651 |
| CoRe | 16.67 | Not recorded | 692d210bd6b0dd3342931651 |

### PS

Pass@5: 90. Result ID: 6995de563badfaffe6e554c2.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 90. Result ID: 6995de563badfaffe6e554c3.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### 0-CoT

Pass@5: 90. Result ID: 6995de563badfaffe6e554c5.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### PS

Pass@5: 90. Result ID: 6995de563badfaffe6e554c6.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 90. Result ID: 6995de563badfaffe6e554c7.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### 0-CoT

Pass@5: 86.67. Result ID: 6995de563badfaffe6e554bd.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### 0-CoT

Pass@5: 86.67. Result ID: 6995de563badfaffe6e554c1.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### PS

Pass@5: 83.33. Result ID: 6995de563badfaffe6e554be.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 83.33. Result ID: 6995de563badfaffe6e554bf.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 83.33. Result ID: 6995de563badfaffe6e554c0.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 4B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 83.33. Result ID: 6995de563badfaffe6e554c4.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 14B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 83.33. Result ID: 6995de563badfaffe6e554c8.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 32B"}. This record does not assert independent reproduction.

### 0-CoT

Pass@5: 30. Result ID: 6995de563badfaffe6e554b9.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6e781e209131cea4017d.md>)

[Implementation](<https://github.com/kojima-takeshi188/zero_shot_cot>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

### PS

Pass@5: 30. Result ID: 6995de563badfaffe6e554ba.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa6f7f1e209131cea45fd1.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

### F-CoT

Pass@5: 26.67. Result ID: 6995de563badfaffe6e554bc.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

### CoRe

Pass@5: 16.67. Result ID: 6995de563badfaffe6e554bb.

[Reporting paper](<https://www.sota2.com/research/paper/692d210bd6b0dd3342931651.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa771d1e209131cea71280.md>)

Recorded evaluation configuration: {"Model":"Qwen 3 0.6B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
