# Mathematical Reasoning on SAT Source: [Mathematical Reasoning on SAT]() Collection index: [llms.txt]() Data updated: 2026-06-09T07:18:32.564Z Evaluation of model performance on the SAT dataset for mathematical reasoning tasks, measured by accuracy. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Reasoning | | Dataset (including any stated split) | SAT | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 35 of 35 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | QAP200 | 98.2 | Not recorded | 68fa749e1e209131cea62a64 | | QAP150 | 97.3 | Not recorded | 68fa749e1e209131cea62a64 | | PS+ | 97.3 | Not recorded | 68fa749e1e209131cea62a64 | | Baseline | 96.8 | Not recorded | 68fa749e1e209131cea62a64 | | QAP100 | 96.8 | Not recorded | 68fa749e1e209131cea62a64 | | TADB | 96.8 | Not recorded | 68fa749e1e209131cea62a64 | | QAP50 | 95.9 | Not recorded | 68fa749e1e209131cea62a64 | | CoT | 95 | Not recorded | 68fa749e1e209131cea62a64 | | QAP25 | 94.5 | Not recorded | 68fa749e1e209131cea62a64 | | Question-Analysis Prompting | 78.6 | Not recorded | 68fa749e1e209131cea62a64 | | PartitionSel | 76.36 | Not recorded | 6a279df490966ade919bf5e5 | | PartitionSel | 75.45 | Not recorded | 6a279df490966ade919bf5e5 | | Question-Analysis Prompting | 75 | Not recorded | 68fa749e1e209131cea62a64 | | Question-Analysis Prompting | 75 | Not recorded | 68fa749e1e209131cea62a64 | | Take A Deep Breath | 74.5 | Not recorded | 68fa749e1e209131cea62a64 | | DoReMi | 72.27 | Not recorded | 6a279df490966ade919bf5e5 | | ID | 71.82 | Not recorded | 6a279df490966ade919bf5e5 | | Baseline | 70.9 | Not recorded | 68fa749e1e209131cea62a64 | | Plan-and-Solve Plus | 70.9 | Not recorded | 68fa749e1e209131cea62a64 | | COLM | 68.64 | Not recorded | 6a279df490966ade919bf5e5 | | IWD | 68.18 | Not recorded | 6a279df490966ade919bf5e5 | | GradNorm | 67.27 | Not recorded | 6a279df490966ade919bf5e5 | | GREATS | 67.27 | Not recorded | 6a279df490966ade919bf5e5 | | Random | 65.94 | Not recorded | 6a279df490966ade919bf5e5 | | Chain-of-Thought | 65.9 | Not recorded | 68fa749e1e209131cea62a64 | | Question-Analysis Prompting | 52.7 | Not recorded | 68fa749e1e209131cea62a64 | | PartitionSel | 44.09 | Not recorded | 6a279df490966ade919bf5e5 | | PartitionSel | 44.09 | Not recorded | 6a279df490966ade919bf5e5 | | GradNorm | 40.91 | Not recorded | 6a279df490966ade919bf5e5 | | GradNorm | 40.91 | Not recorded | 6a279df490966ade919bf5e5 | | COLM | 39.09 | Not recorded | 6a279df490966ade919bf5e5 | | COLM | 39.09 | Not recorded | 6a279df490966ade919bf5e5 | | ID | 38.64 | Not recorded | 6a279df490966ade919bf5e5 | | ID | 38.64 | Not recorded | 6a279df490966ade919bf5e5 | | Question-Analysis Prompting | 35 | Not recorded | 68fa749e1e209131cea62a64 | ### QAP200 Accuracy: 98.2. Result ID: 699363343a6c31484915909e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo","Constraint (N)":200}. This record does not assert independent reproduction. ### QAP150 Accuracy: 97.3. Result ID: 699363343a6c31484915909d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo","Constraint (N)":150}. This record does not assert independent reproduction. ### PS+ Accuracy: 97.3. Result ID: 699363343a6c3148491590a1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo"}. This record does not assert independent reproduction. ### Baseline Accuracy: 96.8. Result ID: 699363343a6c314849159099. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo"}. This record does not assert independent reproduction. ### QAP100 Accuracy: 96.8. Result ID: 699363343a6c31484915909c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo","Constraint (N)":100}. This record does not assert independent reproduction. ### TADB Accuracy: 96.8. Result ID: 699363343a6c31484915909f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo"}. This record does not assert independent reproduction. ### QAP50 Accuracy: 95.9. Result ID: 699363343a6c31484915909b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo","Constraint (N)":50}. This record does not assert independent reproduction. ### CoT Accuracy: 95. Result ID: 699363343a6c3148491590a0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo"}. This record does not assert independent reproduction. ### QAP25 Accuracy: 94.5. Result ID: 699363343a6c31484915909a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-4 Turbo","Constraint (N)":25}. This record does not assert independent reproduction. ### Question-Analysis Prompting Accuracy: 78.6. Result ID: 699363343a6c3148491590b8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Word Count Constraint (n)":"150"}. This record does not assert independent reproduction. ### PartitionSel Accuracy: 76.36. Result ID: 6a27be38fbc373592c241087. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning protocol":"LoRA","Data mixture strategy":"PartitionSel"}. This record does not assert independent reproduction. ### PartitionSel Accuracy: 75.45. Result ID: 6a27be38fbc373592c2410d4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning Dataset":"Meta-MathQA","Subset Fraction":"50%"}. This record does not assert independent reproduction. ### Question-Analysis Prompting Accuracy: 75. Result ID: 699363343a6c3148491590b7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Word Count Constraint (n)":"100"}. This record does not assert independent reproduction. ### Question-Analysis Prompting Accuracy: 75. Result ID: 699363343a6c3148491590b9. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Word Count Constraint (n)":"200"}. This record does not assert independent reproduction. ### Take A Deep Breath Accuracy: 74.5. Result ID: 699363343a6c3148491590ba. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Prompt":"TADB"}. This record does not assert independent reproduction. ### DoReMi Accuracy: 72.27. Result ID: 6a27be38fbc373592c241086. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning protocol":"LoRA","Data mixture strategy":"DoReMi"}. This record does not assert independent reproduction. ### ID Accuracy: 71.82. Result ID: 6a27be38fbc373592c2410d2. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning Dataset":"Meta-MathQA","Subset Fraction":"50%"}. This record does not assert independent reproduction. ### Baseline Accuracy: 70.9. Result ID: 699363343a6c3148491590b4. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Prompt":"Baseline"}. This record does not assert independent reproduction. ### Plan-and-Solve Plus Accuracy: 70.9. Result ID: 699363343a6c3148491590bc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Prompt":"PS+"}. This record does not assert independent reproduction. ### COLM Accuracy: 68.64. Result ID: 6a27be38fbc373592c2410d0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning Dataset":"Meta-MathQA","Subset Fraction":"50%"}. This record does not assert independent reproduction. ### IWD Accuracy: 68.18. Result ID: 6a27be38fbc373592c2410d3. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning Dataset":"Meta-MathQA","Subset Fraction":"50%"}. This record does not assert independent reproduction. ### GradNorm Accuracy: 67.27. Result ID: 6a27be38fbc373592c2410cf. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning Dataset":"Meta-MathQA","Subset Fraction":"50%"}. This record does not assert independent reproduction. ### GREATS Accuracy: 67.27. Result ID: 6a27be38fbc373592c2410d1. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning Dataset":"Meta-MathQA","Subset Fraction":"50%"}. This record does not assert independent reproduction. ### Random Accuracy: 65.94. Result ID: 6a27be38fbc373592c2410ce. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen2.5-3B","Fine-tuning Dataset":"Meta-MathQA","Subset Fraction":"50%"}. This record does not assert independent reproduction. ### Chain-of-Thought Accuracy: 65.9. Result ID: 699363343a6c3148491590bb. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Prompt":"CoT"}. This record does not assert independent reproduction. ### Question-Analysis Prompting Accuracy: 52.7. Result ID: 699363343a6c3148491590b6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Word Count Constraint (n)":"50"}. This record does not assert independent reproduction. ### PartitionSel Accuracy: 44.09. Result ID: 6a27be38fbc373592c2410f4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### PartitionSel Accuracy: 44.09. Result ID: 6a27be38fbc373592c241153. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### GradNorm Accuracy: 40.91. Result ID: 6a27be38fbc373592c2410f2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### GradNorm Accuracy: 40.91. Result ID: 6a27be38fbc373592c241151. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### COLM Accuracy: 39.09. Result ID: 6a27be38fbc373592c2410f1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### COLM Accuracy: 39.09. Result ID: 6a27be38fbc373592c241150. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### ID Accuracy: 38.64. Result ID: 6a27be38fbc373592c2410f3. [Reporting paper]() Recorded evaluation configuration: {"Base Model":"Llama-3.2-3B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### ID Accuracy: 38.64. Result ID: 6a27be38fbc373592c241152. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Llama-3.1-8B","Training Dataset":"MetaMathQA","Subset Ratio":"12.5%"}. This record does not assert independent reproduction. ### Question-Analysis Prompting Accuracy: 35. Result ID: 699363343a6c3148491590b5. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT-3.5 Turbo","Word Count Constraint (n)":"25"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.