# Mathematical Reasoning on AIME 24 (pass@4) Source: [Mathematical Reasoning on AIME 24 (pass@4)]() Collection index: [llms.txt]() Data updated: 2026-05-20T07:03:31.740Z Evaluation of model performance on the AIME 2024 dataset for mathematical reasoning, specifically reporting the pass@4 metric. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | AIME 24 | | Metric | Pass@4 | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 25 of 25 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Pass@4](): Higher is better; unit: PERCENT. ## Recorded results | Method | Pass@4 | Result date | Report paper | | --- | --- | --- | --- | | Hybrid-LoRA | 60 | Not recorded | 6a0d4012f984739ed67377be | | Hybrid-LoRA | 56 | Not recorded | 6a0d4012f984739ed67377be | | AutoLoRA | 55.3 | Not recorded | 6a0d4012f984739ed67377be | | AutoLoRA | 52.7 | Not recorded | 6a0d4012f984739ed67377be | | Full-FT (ref.) | 49.3 | Not recorded | 6a0d4012f984739ed67377be | | Hybrid-LoRA | 48 | Not recorded | 6a0d4012f984739ed67377be | | Full-FT (ref.) | 46 | Not recorded | 6a0d4012f984739ed67377be | | AutoLoRA | 44 | Not recorded | 6a0d4012f984739ed67377be | | Hybrid-LoRA | 43.3 | Not recorded | 6a0d4012f984739ed67377be | | Full-FT (ref.) | 42 | Not recorded | 6a0d4012f984739ed67377be | | Hybrid-LoRA | 42 | Not recorded | 6a0d4012f984739ed67377be | | AutoLoRA | 40.7 | Not recorded | 6a0d4012f984739ed67377be | | Full-FT (ref.) | 40 | Not recorded | 6a0d4012f984739ed67377be | | LoRA-Drop | 40 | Not recorded | 6a0d4012f984739ed67377be | | Hybrid-LoRA | 38 | Not recorded | 6a0d4012f984739ed67377be | | AdaLoRA | 36 | Not recorded | 6a0d4012f984739ed67377be | | AutoLoRA | 35.3 | Not recorded | 6a0d4012f984739ed67377be | | AutoLoRA | 34.7 | Not recorded | 6a0d4012f984739ed67377be | | LoRA-Drop | 34 | Not recorded | 6a0d4012f984739ed67377be | | LoRA-FFN | 34 | Not recorded | 6a0d4012f984739ed67377be | | AdaLoRA | 32 | Not recorded | 6a0d4012f984739ed67377be | | CoT | 24 | Not recorded | 6a0d4012f984739ed67377be | | Full-FT | 22.7 | Not recorded | 6a0d4012f984739ed67377be | | COT | 14.7 | Not recorded | 6a0d4012f984739ed67377be | | IO | 3.3 | Not recorded | 6a0d4012f984739ed67377be | ### Hybrid-LoRA Pass@4: 60. Result ID: 6a0d5c4efbc373592c04e6a6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 7B Instruct","Post-training Approach":"GSPO","#Tunable Params":"449M"}. This record does not assert independent reproduction. ### Hybrid-LoRA Pass@4: 56. Result ID: 6a0d5c4efbc373592c04e6a4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 7B Instruct","Post-training Approach":"GRPO","#Tunable Params":"449M"}. This record does not assert independent reproduction. ### AutoLoRA Pass@4: 55.3. Result ID: 6a0d5c4efbc373592c04e6a5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 7B Instruct","Post-training Approach":"GSPO","#Tunable Params":"453M"}. This record does not assert independent reproduction. ### AutoLoRA Pass@4: 52.7. Result ID: 6a0d5c4efbc373592c04e6a3. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 7B Instruct","Post-training Approach":"GRPO","#Tunable Params":"453M"}. This record does not assert independent reproduction. ### Full-FT (ref.) Pass@4: 49.3. Result ID: 6a0d5c4efbc373592c04e6a0. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 3B Instruct","Post-training Approach":"GSPO","#Tunable Params":"3086M"}. This record does not assert independent reproduction. ### Hybrid-LoRA Pass@4: 48. Result ID: 6a0d5c4efbc373592c04e6a2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 3B Instruct","Post-training Approach":"GSPO","#Tunable Params":"192M"}. This record does not assert independent reproduction. ### Full-FT (ref.) Pass@4: 46. Result ID: 6a0d5c4efbc373592c04e69d. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 3B Instruct","Post-training Approach":"GRPO","#Tunable Params":"3086M"}. This record does not assert independent reproduction. ### AutoLoRA Pass@4: 44. Result ID: 6a0d5c4efbc373592c04e6a1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 3B Instruct","Post-training Approach":"GSPO","#Tunable Params":"194M"}. This record does not assert independent reproduction. ### Hybrid-LoRA Pass@4: 43.3. Result ID: 6a0d5c4efbc373592c04e69f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 3B Instruct","Post-training Approach":"GRPO","#Tunable Params":"192M"}. This record does not assert independent reproduction. ### Full-FT (ref.) Pass@4: 42. Result ID: 6a0d5c4efbc373592c04e696. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GSPO","#Tunable Params":"1544M"}. This record does not assert independent reproduction. ### Hybrid-LoRA Pass@4: 42. Result ID: 6a0d5c4efbc373592c04e69b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GSPO","#Tunable Params":"105M"}. This record does not assert independent reproduction. ### AutoLoRA Pass@4: 40.7. Result ID: 6a0d5c4efbc373592c04e69e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 3B Instruct","Post-training Approach":"GRPO","#Tunable Params":"201M"}. This record does not assert independent reproduction. ### Full-FT (ref.) Pass@4: 40. Result ID: 6a0d5c4efbc373592c04e691. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GRPO","#Tunable Params":"1544M"}. This record does not assert independent reproduction. ### LoRA-Drop Pass@4: 40. Result ID: 6a0d5c4efbc373592c04e69a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GSPO","#Tunable Params":"105M"}. This record does not assert independent reproduction. ### Hybrid-LoRA Pass@4: 38. Result ID: 6a0d5c4efbc373592c04e695. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GRPO","#Tunable Params":"105M"}. This record does not assert independent reproduction. ### AdaLoRA Pass@4: 36. Result ID: 6a0d5c4efbc373592c04e698. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GSPO","#Tunable Params":"106M"}. This record does not assert independent reproduction. ### AutoLoRA Pass@4: 35.3. Result ID: 6a0d5c4efbc373592c04e699. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GSPO","#Tunable Params":"107M"}. This record does not assert independent reproduction. ### AutoLoRA Pass@4: 34.7. Result ID: 6a0d5c4efbc373592c04e693. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GRPO","#Tunable Params":"106M"}. This record does not assert independent reproduction. ### LoRA-Drop Pass@4: 34. Result ID: 6a0d5c4efbc373592c04e694. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GRPO","#Tunable Params":"105M"}. This record does not assert independent reproduction. ### LoRA-FFN Pass@4: 34. Result ID: 6a0d5c4efbc373592c04e697. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GSPO","#Tunable Params":"122M"}. This record does not assert independent reproduction. ### AdaLoRA Pass@4: 32. Result ID: 6a0d5c4efbc373592c04e692. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"GRPO","#Tunable Params":"105M"}. This record does not assert independent reproduction. ### CoT Pass@4: 24. Result ID: 6a0d5c4efbc373592c04e69c. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 3B Instruct","Post-training Approach":"-","#Tunable Params":"0.0"}. This record does not assert independent reproduction. ### Full-FT Pass@4: 22.7. Result ID: 6a0d5c4efbc373592c04e690. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"SFT","#Tunable Params":"1544M"}. This record does not assert independent reproduction. ### COT Pass@4: 14.7. Result ID: 6a0d5c4efbc373592c04e68f. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"COT","#Tunable Params":"0.0"}. This record does not assert independent reproduction. ### IO Pass@4: 3.3. Result ID: 6a0d5c4efbc373592c04e68e. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen-2.5 1.5B Instruct","Post-training Approach":"IO","#Tunable Params":"0.0"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.