Arithmetic Reasoning on GSM8K (n=200 test subset)
95.5AccuracyChain-of-Thought Prompting
Evaluation Results
| Method | Links | |
|---|---|---|
| Chain-of-Thought PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 95.5 | |
| Plan & Solve PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 95.5 | |
| Trace-of-Thought PromptingBase Model=GPT-4, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 95 | |
| Standard PromptingBase Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 94.5 | |
| Trace-of-Thought PromptingBase Model=Llama 3 Chat 8B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 88 | |
| Trace-of-Thought PromptingBase Model=GPT-3.5-Turbo, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 86.5 | |
| Trace-of-Thought PromptingBase Model=GPT-4, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 83 | |
| Plan & Solve PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.04 | 82.5 | |
| Trace-of-Thought PromptingBase Model=WizardMath-7B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 81.5 | |
| Standard PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.04 | 75.5 | |
| Plan & Solve PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.04 | 74.5 | |
| Chain-of-Thought PromptingBase Model=GPT-3.5-Turbo, Prompting Type=Zero-Shot2025.04 | 73.5 | |
| Chain-of-Thought PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.04 | 73.5 | |
| Standard PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.04 | 73 | |
| Chain-of-Thought PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.04 | 73 | |
| Trace-of-Thought PromptingBase Model=WizardMath-7B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 70.5 | |
| Standard PromptingBase Model=WizardMath-7B, Prompting Type=Zero-Shot2025.04 | 69 | |
| Plan & Solve PromptingBase Model=Llama 3 Chat 8B, Prompting Type=Zero-Shot2025.04 | 68.5 | |
| Trace-of-Thought PromptingBase Model=GPT-3.5-Turbo, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 64.5 | |
| Trace-of-Thought PromptingBase Model=Llama 3 Chat 8B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 63.5 | |
| Trace-of-Thought PromptingBase Model=Zephyr, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 55 | |
| Trace-of-Thought PromptingBase Model=Llama 2 Chat 7B, Teacher Model=GPT-4, Prompting Type=Zero-Shot2025.04 | 50 | |
| Trace-of-Thought PromptingBase Model=Zephyr, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 43 | |
| Trace-of-Thought PromptingBase Model=Llama 2 Chat 7B, Teacher Model=Llama 3 8B, Prompting Type=Zero-Shot2025.04 | 37.5 | |
| Plan & Solve PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.04 | 30 | |
| Standard PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.04 | 26 | |
| Chain-of-Thought PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.04 | 23.5 | |
| Chain-of-Thought PromptingBase Model=Zephyr, Prompting Type=Zero-Shot2025.04 | 23.5 | |
| Plan & Solve PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.04 | 23 | |
| Standard PromptingBase Model=Llama 2 Chat 7B, Prompting Type=Zero-Shot2025.04 | 22 |