# Competitive Mathematical Reasoning on AIME 2024 (Pass@1) Source: [Competitive Mathematical Reasoning on AIME 2024 (Pass@1)]() Collection index: [llms.txt]() Data updated: 2026-06-09T10:04:07.631Z Evaluation of model performance on the American Invitational Mathematics Examination (AIME) 2024 dataset, specifically measuring success rate with a single sample (Pass@1). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Competitive Mathematical Reasoning | | Dataset (including any stated split) | AIME 2024 | | Metric | Pass@1 Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 37 of 37 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@1 Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@1 Accuracy | Result date | Report paper | | --- | --- | --- | --- | | DeepSeek-R1 | 100 | Not recorded | 6a279e7c90966ade919bfbd0 | | GPT-5(high) | 96.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | DeepSeek-V4-pro | 96.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | Qwen3-max | 93.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | DiScOw/o ITDE | 86.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | DiScO | 86.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | Qwen3-32B | 83.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | DeepSeek-R1-Distill-Qwen-32B | 80 | Not recorded | 6a279e7c90966ade919bfbd0 | | DiScO | 70 | Not recorded | 6a279e7c90966ade919bfbd0 | | DiScOw/o ITDE | 66.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | GPT-o1-mini | 56.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | Qwen3-8B | 56.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | ReasonFlux-32B | 56.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | rStar-Math-7B | 50 | Not recorded | 6a279e7c90966ade919bfbd0 | | QwQ-32B-preview | 50 | Not recorded | 6a279e7c90966ade919bfbd0 | | DeepSeek-V3 | 47.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | GPT-o1-preview | 44.6 | Not recorded | 6a279e7c90966ade919bfbd0 | | Sky-T1-32B-preview | 43.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | DeepSeek-R1-Distill-Qwen-7B | 36.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | ReasonFlux-7B | 36.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | Qwen2.5-Math-72B-Instruct | 30 | Not recorded | 6a279e7c90966ade919bfbd0 | | SuperCorrect-7B | 26.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | DIVER | 25 | Not recorded | 6a279e7c90966ade919bfbd0 | | LLaMA3.1-405B-Instruct | 23.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | Entropy-RL | 23.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | GRPO w/ Clip-higher | 20 | Not recorded | 6a279e7c90966ade919bfbd0 | | Pass@k Training | 20 | Not recorded | 6a279e7c90966ade919bfbd0 | | GPT-4o | 16.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | LLaMA3.1-70B-Instruct | 16.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | Qwen2.5-Math-7B | 16.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | Qwen2.5-32B-Instruct | 16.5 | Not recorded | 6a279e7c90966ade919bfbd0 | | Claude3.5-Sonnet | 16 | Not recorded | 6a279e7c90966ade919bfbd0 | | DeepSeek-Coder-V2-Instruct | 13.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | Qwen2.5-Math-7B-Instruct | 13.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | LLaMA3.1-8B-Instruct | 6.7 | Not recorded | 6a279e7c90966ade919bfbd0 | | NuminaMath-72B-CoT | 3.3 | Not recorded | 6a279e7c90966ade919bfbd0 | | Mathstral-7B-v0.1 | 0 | Not recorded | 6a279e7c90966ade919bfbd0 | ### DeepSeek-R1 Pass@1 Accuracy: 100. Result ID: 6a27e45dfbc373592c25300a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### GPT-5(high) Pass@1 Accuracy: 96.7. Result ID: 6a27e45dfbc373592c253003. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Closed-source frontier"}. This record does not assert independent reproduction. ### DeepSeek-V4-pro Pass@1 Accuracy: 96.7. Result ID: 6a27e45dfbc373592c25300b. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### Qwen3-max Pass@1 Accuracy: 93.3. Result ID: 6a27e45dfbc373592c253002. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Closed-source frontier"}. This record does not assert independent reproduction. ### DiScOw/o ITDE Pass@1 Accuracy: 86.7. Result ID: 6a27e45dfbc373592c253021. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Ours"}. This record does not assert independent reproduction. ### DiScO Pass@1 Accuracy: 86.7. Result ID: 6a27e45dfbc373592c253022. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Ours"}. This record does not assert independent reproduction. ### Qwen3-32B Pass@1 Accuracy: 83.3. Result ID: 6a27e45dfbc373592c253020. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### DeepSeek-R1-Distill-Qwen-32B Pass@1 Accuracy: 80. Result ID: 6a27e45dfbc373592c25301f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### DiScO Pass@1 Accuracy: 70. Result ID: 6a27e45dfbc373592c25301a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Ours"}. This record does not assert independent reproduction. ### DiScOw/o ITDE Pass@1 Accuracy: 66.7. Result ID: 6a27e45dfbc373592c253019. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Ours"}. This record does not assert independent reproduction. ### GPT-o1-mini Pass@1 Accuracy: 56.7. Result ID: 6a27e45dfbc373592c253000. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Closed-source frontier"}. This record does not assert independent reproduction. ### Qwen3-8B Pass@1 Accuracy: 56.7. Result ID: 6a27e45dfbc373592c253013. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### ReasonFlux-32B Pass@1 Accuracy: 56.7. Result ID: 6a27e45dfbc373592c25301d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### rStar-Math-7B Pass@1 Accuracy: 50. Result ID: 6a27e45dfbc373592c253014. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### QwQ-32B-preview Pass@1 Accuracy: 50. Result ID: 6a27e45dfbc373592c25301e. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### DeepSeek-V3 Pass@1 Accuracy: 47.7. Result ID: 6a27e45dfbc373592c253009. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### GPT-o1-preview Pass@1 Accuracy: 44.6. Result ID: 6a27e45dfbc373592c253001. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Closed-source frontier"}. This record does not assert independent reproduction. ### Sky-T1-32B-preview Pass@1 Accuracy: 43.3. Result ID: 6a27e45dfbc373592c25301c. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### DeepSeek-R1-Distill-Qwen-7B Pass@1 Accuracy: 36.7. Result ID: 6a27e45dfbc373592c253011. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### ReasonFlux-7B Pass@1 Accuracy: 36.7. Result ID: 6a27e45dfbc373592c253012. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### Qwen2.5-Math-72B-Instruct Pass@1 Accuracy: 30. Result ID: 6a27e45dfbc373592c253008. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### SuperCorrect-7B Pass@1 Accuracy: 26.7. Result ID: 6a27e45dfbc373592c253010. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Reasoning-specialized models"}. This record does not assert independent reproduction. ### DIVER Pass@1 Accuracy: 25. Result ID: 6a27e45dfbc373592c253018. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Diversity-oriented RL models"}. This record does not assert independent reproduction. ### LLaMA3.1-405B-Instruct Pass@1 Accuracy: 23.3. Result ID: 6a27e45dfbc373592c253007. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### Entropy-RL Pass@1 Accuracy: 23.3. Result ID: 6a27e45dfbc373592c253015. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Diversity-oriented RL models"}. This record does not assert independent reproduction. ### GRPO w/ Clip-higher Pass@1 Accuracy: 20. Result ID: 6a27e45dfbc373592c253016. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Diversity-oriented RL models"}. This record does not assert independent reproduction. ### Pass@k Training Pass@1 Accuracy: 20. Result ID: 6a27e45dfbc373592c253017. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Diversity-oriented RL models"}. This record does not assert independent reproduction. ### GPT-4o Pass@1 Accuracy: 16.7. Result ID: 6a27e45dfbc373592c252ffe. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Closed-source frontier"}. This record does not assert independent reproduction. ### LLaMA3.1-70B-Instruct Pass@1 Accuracy: 16.7. Result ID: 6a27e45dfbc373592c253005. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### Qwen2.5-Math-7B Pass@1 Accuracy: 16.7. Result ID: 6a27e45dfbc373592c25300e. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Math instruction-tuned models"}. This record does not assert independent reproduction. ### Qwen2.5-32B-Instruct Pass@1 Accuracy: 16.5. Result ID: 6a27e45dfbc373592c25301b. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"32B","Model Category":"Instruction-tuned models"}. This record does not assert independent reproduction. ### Claude3.5-Sonnet Pass@1 Accuracy: 16. Result ID: 6a27e45dfbc373592c252fff. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Closed-source frontier"}. This record does not assert independent reproduction. ### DeepSeek-Coder-V2-Instruct Pass@1 Accuracy: 13.3. Result ID: 6a27e45dfbc373592c253006. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### Qwen2.5-Math-7B-Instruct Pass@1 Accuracy: 13.3. Result ID: 6a27e45dfbc373592c25300f. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"Math instruction-tuned models"}. This record does not assert independent reproduction. ### LLaMA3.1-8B-Instruct Pass@1 Accuracy: 6.7. Result ID: 6a27e45dfbc373592c25300c. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"General instruction-tuned models"}. This record does not assert independent reproduction. ### NuminaMath-72B-CoT Pass@1 Accuracy: 3.3. Result ID: 6a27e45dfbc373592c253004. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"Reference","Model Category":"Open-source frontier"}. This record does not assert independent reproduction. ### Mathstral-7B-v0.1 Pass@1 Accuracy: 0. Result ID: 6a27e45dfbc373592c25300d. [Reporting paper]() Recorded evaluation configuration: {"Model Scale":"7B","Model Category":"General instruction-tuned models"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.