List your company
ResearchFormerly WizWand

Math Reasoning on MATH 200 samples (test)

75AccuracyPlan & Solve Prompting
2.221.14058.9Apr 29, 2025May 9, 2025May 20, 2025May 31, 2025Jun 11, 2025Jun 22, 2025Jul 3, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Plan & Solve PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.0475—
Trace-of-Thought PromptingBase Model=GPT-4, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.0468—
Chain-of-Thought PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.0466—
CNTP + SC (Ours + SC)Model=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Stochastic, paths=402025.0757.534.5
Standard PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.0457.5—
SCModel=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Stochastic, paths=402025.075611.7
Plan & Solve PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.0456—
Trace-of-Thought PromptingBase Model=GPT-3.5-Turbo, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.0456—
Trace-of-Thought PromptingBase Model=GPT-4, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.0455—
Chain-of-Thought PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.0452—
Beam SearchModel=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Deterministic, beam=52025.07481.46
CNTP (Ours*)Model=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Stochastic2025.0747.10.87
Standard PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.0446.5—
Standard PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.0444.5—
Trace-of-Thought PromptingBase Model=WizardMath-7B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.0442.5—
Greedy DecodingModel=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Deterministic2025.0741.50.29
Stochastic Decoding*Model=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Stochastic2025.0741.50.29
Trace-of-Thought PromptingBase Model=Llama 3 Chat 8B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.0441—
Trace-of-Thought PromptingBase Model=GPT-3.5-Turbo, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.0440.5—
Plan & Solve PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.0437—
Chain-of-Thought PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.0435.5—
Chain-of-Thought PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.0433.5—
Trace-of-Thought PromptingBase Model=WizardMath-7B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.0430.5—
Standard PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.0430.5—
Plan & Solve PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.0430—
Trace-of-Thought PromptingBase Model=Llama 3 Chat 8B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.0423—
Trace-of-Thought PromptingBase Model=Zephyr, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.0414.5—
Trace-of-Thought PromptingBase Model=Zephyr, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.0413.5—
Chain-of-Thought PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.0412—
Plan & Solve PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.049—
Trace-of-Thought PromptingBase Model=Llama 2 Chat 7B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.048—
Chain-of-Thought PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.047—
Standard PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.047—
Standard PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.046.5—
Trace-of-Thought PromptingBase Model=Llama 2 Chat 7B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.046.5—
Plan & Solve PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.045—