Mathematical Reasoning on SAT
98.2AccuracyQAP200
Evaluation Results
| Method | Links | |
|---|---|---|
| QAP200Model=GPT-4 Turbo, Constraint (N)=2002024.07 | 98.2 | |
| QAP150Model=GPT-4 Turbo, Constraint (N)=1502024.07 | 97.3 | |
| PS+Model=GPT-4 Turbo2024.07 | 97.3 | |
| BaselineModel=GPT-4 Turbo2024.07 | 96.8 | |
| QAP100Model=GPT-4 Turbo, Constraint (N)=1002024.07 | 96.8 | |
| TADBModel=GPT-4 Turbo2024.07 | 96.8 | |
| QAP50Model=GPT-4 Turbo, Constraint (N)=502024.07 | 95.9 | |
| CoTModel=GPT-4 Turbo2024.07 | 95 | |
| QAP25Model=GPT-4 Turbo, Constraint (N)=252024.07 | 94.5 | |
| Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=1502024.07 | 78.6 | |
| PartitionSelBackbone=Qwen2.5-3B, Fine-tuning protocol=LoRA, Data mixture strategy=PartitionSel2026.06 | 76.36 | |
| PartitionSelBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.06 | 75.45 | |
| Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=1002024.07 | 75 | |
| Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=2002024.07 | 75 | |
| Take A Deep BreathModel=GPT-3.5 Turbo, Prompt=TADB2024.07 | 74.5 | |
| DoReMiBackbone=Qwen2.5-3B, Fine-tuning protocol=LoRA, Data mixture strategy=DoReMi2026.06 | 72.27 | |
| IDBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.06 | 71.82 | |
| BaselineModel=GPT-3.5 Turbo, Prompt=Baseline2024.07 | 70.9 | |
| Plan-and-Solve PlusModel=GPT-3.5 Turbo, Prompt=PS+2024.07 | 70.9 | |
| COLMBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.06 | 68.64 | |
| IWDBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.06 | 68.18 | |
| GradNormBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.06 | 67.27 | |
| GREATSBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.06 | 67.27 | |
| RandomBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.06 | 65.94 | |
| Chain-of-ThoughtModel=GPT-3.5 Turbo, Prompt=CoT2024.07 | 65.9 | |
| Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=502024.07 | 52.7 | |
| PartitionSelBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 44.09 | |
| PartitionSelBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 44.09 | |
| GradNormBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 40.91 | |
| GradNormBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 40.91 | |
| COLMBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 39.09 | |
| COLMBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 39.09 | |
| IDBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 38.64 | |
| IDBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.06 | 38.64 | |
| Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=252024.07 | 35 |