Math Reasoning on MATH 200 samples (test)
75AccuracyPlan & Solve Prompting
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Plan & Solve PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 75 | — | |
| Trace-of-Thought PromptingBase Model=GPT-4, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 68 | — | |
| Chain-of-Thought PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 66 | — | |
| CNTP + SC (Ours + SC)Model=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Stochastic, paths=402025.07 | 57.5 | 34.5 | |
| Standard PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 57.5 | — | |
| SCModel=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Stochastic, paths=402025.07 | 56 | 11.7 | |
| Plan & Solve PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.04 | 56 | — | |
| Trace-of-Thought PromptingBase Model=GPT-3.5-Turbo, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 56 | — | |
| Trace-of-Thought PromptingBase Model=GPT-4, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 55 | — | |
| Chain-of-Thought PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.04 | 52 | — | |
| Beam SearchModel=Llama-3.1-8B-Instruct, Reasoning Chain=Multiple Reasoning Chains, Decoding Strategy=Deterministic, beam=52025.07 | 48 | 1.46 | |
| CNTP (Ours*)Model=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Stochastic2025.07 | 47.1 | 0.87 | |
| Standard PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.04 | 46.5 | — | |
| Standard PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.04 | 44.5 | — | |
| Trace-of-Thought PromptingBase Model=WizardMath-7B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 42.5 | — | |
| Greedy DecodingModel=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Deterministic2025.07 | 41.5 | 0.29 | |
| Stochastic Decoding*Model=Llama-3.1-8B-Instruct, Reasoning Chain=Single Reasoning Chain, Decoding Strategy=Stochastic2025.07 | 41.5 | 0.29 | |
| Trace-of-Thought PromptingBase Model=Llama 3 Chat 8B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 41 | — | |
| Trace-of-Thought PromptingBase Model=GPT-3.5-Turbo, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 40.5 | — | |
| Plan & Solve PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.04 | 37 | — | |
| Chain-of-Thought PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.04 | 35.5 | — | |
| Chain-of-Thought PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.04 | 33.5 | — | |
| Trace-of-Thought PromptingBase Model=WizardMath-7B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 30.5 | — | |
| Standard PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.04 | 30.5 | — | |
| Plan & Solve PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.04 | 30 | — | |
| Trace-of-Thought PromptingBase Model=Llama 3 Chat 8B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 23 | — | |
| Trace-of-Thought PromptingBase Model=Zephyr, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 14.5 | — | |
| Trace-of-Thought PromptingBase Model=Zephyr, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 13.5 | — | |
| Chain-of-Thought PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.04 | 12 | — | |
| Plan & Solve PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.04 | 9 | — | |
| Trace-of-Thought PromptingBase Model=Llama 2 Chat 7B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 8 | — | |
| Chain-of-Thought PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.04 | 7 | — | |
| Standard PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.04 | 7 | — | |
| Standard PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.04 | 6.5 | — | |
| Trace-of-Thought PromptingBase Model=Llama 2 Chat 7B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 6.5 | — | |
| Plan & Solve PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.04 | 5 | — |