Symbolic Reasoning on Coin Flip
99.9AccuracyAuto-CoT
Evaluation results
| Method | Links | |
|---|---|---|
| Auto-CoTPrompting=Automatic Chain-of-Thought2022.10 | 99.9 | |
| Auto-CoTBase Model=text-davinci-0022023.06 | 99.9 | |
| CoK + SCBase Model=gpt-3.5-turbo2023.06 | 99.2 | |
| Manual CoT + SCBase Model=gpt-3.5-turbo2023.06 | 99 | |
| CoKBase Model=gpt-3.5-turbo2023.06 | 98 | |
| CoKBase Model=text-davinci-0022023.06 | 97.4 | |
| Manual CoTBase Model=gpt-3.5-turbo2023.06 | 97.4 | |
| Manual-CoTPrompting=Manual Chain-of-Thought2022.10 | 97.2 | |
| Zero-Shot-CoTPrompting=Zero-shot Chain-of-Thought2022.10 | 91.4 | |
| Zero-Shot CoTBase Model=text-davinci-0022023.06 | 91.4 | |
| RM-RegenModel=GPT-3.52026.03 | 78.5 | |
| RM-RegenModel=Llama3.1-8B2026.03 | 77.75 | |
| ProCoModel=GPT-3.5, iterations=32026.03 | 75.25 | |
| ST CoTModel=GPT-3.5, iterations=32026.03 | 75 | |
| Manual CoTBase Model=text-davinci-0022023.06 | 74.5 | |
| ST CoTModel=Llama3.1-8B, iterations=32026.03 | 69.25 | |
| Self-RefineModel=GPT-3.5, iterations=32026.03 | 68.25 | |
| Self-RefineModel=Llama3.1-8B, iterations=32026.03 | 66 | |
| RM-RegenModel=Gemma2-9B2026.03 | 66 | |
| Few-ShotPrompting=Few-shot2022.10 | 57.2 | |
| Self-RefineModel=Gemma2-9B, iterations=32026.03 | 54.25 | |
| Zero-ShotPrompting=Zero-shot2022.10 | 53.8 | |
| ProCoModel=Gemma2-9B, iterations=32026.03 | 50.75 | |
| Few-Shot SPBase Model=text-davinci-0022023.06 | 49.1 | |
| ST CoTModel=Gemma2-9B, iterations=32026.03 | 45 | |
| ProCoModel=Llama3.1-8B, iterations=32026.03 | 44 | |
| Zero-Shot SPBase Model=text-davinci-0022023.06 | 12.8 |