List your company
ResearchFormerly WizWand

Mathematical Reasoning on SAT

98.2AccuracyQAP200
32.47249.53666.683.664Jul 4, 2024Oct 29, 2024Feb 23, 2025Jun 20, 2025Oct 15, 2025Feb 9, 2026Jun 6, 2026
Updated 5mo ago

Evaluation results

MethodLinks
QAP200Model=GPT-4 Turbo, Constraint (N)=2002024.0798.2
QAP150Model=GPT-4 Turbo, Constraint (N)=1502024.0797.3
PS+Model=GPT-4 Turbo2024.0797.3
BaselineModel=GPT-4 Turbo2024.0796.8
QAP100Model=GPT-4 Turbo, Constraint (N)=1002024.0796.8
TADBModel=GPT-4 Turbo2024.0796.8
QAP50Model=GPT-4 Turbo, Constraint (N)=502024.0795.9
CoTModel=GPT-4 Turbo2024.0795
QAP25Model=GPT-4 Turbo, Constraint (N)=252024.0794.5
Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=1502024.0778.6
PartitionSelBackbone=Qwen2.5-3B, Fine-tuning protocol=LoRA, Data mixture strategy=PartitionSel2026.0676.36
PartitionSelBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.0675.45
Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=1002024.0775
Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=2002024.0775
Take A Deep BreathModel=GPT-3.5 Turbo, Prompt=TADB2024.0774.5
DoReMiBackbone=Qwen2.5-3B, Fine-tuning protocol=LoRA, Data mixture strategy=DoReMi2026.0672.27
IDBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.0671.82
BaselineModel=GPT-3.5 Turbo, Prompt=Baseline2024.0770.9
Plan-and-Solve PlusModel=GPT-3.5 Turbo, Prompt=PS+2024.0770.9
COLMBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.0668.64
IWDBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.0668.18
GradNormBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.0667.27
GREATSBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.0667.27
RandomBackbone=Qwen2.5-3B, Fine-tuning Dataset=Meta-MathQA, Subset Fraction=50%2026.0665.94
Chain-of-ThoughtModel=GPT-3.5 Turbo, Prompt=CoT2024.0765.9
Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=502024.0752.7
PartitionSelBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0644.09
PartitionSelBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0644.09
GradNormBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0640.91
GradNormBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0640.91
COLMBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0639.09
COLMBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0639.09
IDBase Model=Llama-3.2-3B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0638.64
IDBackbone=Llama-3.1-8B, Training Dataset=MetaMathQA, Subset Ratio=12.5%2026.0638.64
Question-Analysis PromptingModel=GPT-3.5 Turbo, Word Count Constraint (n)=252024.0735