Math Word Problem Solving on GSM8K (train/dev)
88.3AccuracyCoT
Evaluation results
| Method | Links | |
|---|---|---|
| CoTCompilation=+ensemble, Training=trainset, Model=GPT-3.52023.10 | 88.3 | |
| reflectionCompilation=+ensemble, Training=trainset, Model=GPT-3.52023.10 | 86.7 | |
| reflectionCompilation=bootstrap, Training=trainset, Model=GPT-3.52023.10 | 83 | |
| CoTCompilation=bootstrap, Training=trainset, Model=GPT-3.52023.10 | 80.3 | |
| CoTCompilation=fewshot, Training=+human_CoT, Model=GPT-3.52023.10 | 78.6 | |
| reflectionCompilation=fewshot, Training=trainset, Model=GPT-3.52023.10 | 71.7 | |
| reflectionCompilation=none, Training=n/a, Model=GPT-3.52023.10 | 65 | |
| vanillaCompilation=bootstrap×2, Training=trainset, Model=GPT-3.52023.10 | 64.7 | |
| CoTCompilation=fewshot, Training=trainset, Model=GPT-3.52023.10 | 63 | |
| vanillaCompilation=+ensemble, Training=trainset, Model=GPT-3.52023.10 | 62.7 | |
| CoTCompilation=none, Training=n/a, Model=GPT-3.52023.10 | 50 | |
| reflectionCompilation=+ensemble, Training=trainset, Model=Llama2-13b-chat2023.10 | 49 | |
| reflectionCompilation=bootstrap, Training=trainset, Model=Llama2-13b-chat2023.10 | 44.3 | |
| vanillaCompilation=bootstrap, Training=trainset, Model=GPT-3.52023.10 | 44 | |
| CoTCompilation=+ensemble, Training=trainset, Model=Llama2-13b-chat2023.10 | 43.7 | |
| CoTCompilation=bootstrap, Training=trainset, Model=Llama2-13b-chat2023.10 | 43.3 | |
| vanillaCompilation=+ensemble, Training=trainset, Model=Llama2-13b-chat2023.10 | 39 | |
| vanillaCompilation=bootstrap×2, Training=trainset, Model=Llama2-13b-chat2023.10 | 37.3 | |
| reflectionCompilation=none, Training=n/a, Model=Llama2-13b-chat2023.10 | 36.7 | |
| reflectionCompilation=fewshot, Training=trainset, Model=Llama2-13b-chat2023.10 | 36.3 | |
| CoTCompilation=fewshot, Training=+human_CoT, Model=Llama2-13b-chat2023.10 | 34.3 | |
| vanillaCompilation=fewshot, Training=trainset, Model=GPT-3.52023.10 | 33.1 | |
| vanillaCompilation=bootstrap, Training=trainset, Model=Llama2-13b-chat2023.10 | 28 | |
| CoTCompilation=fewshot, Training=trainset, Model=Llama2-13b-chat2023.10 | 27.3 | |
| CoTCompilation=none, Training=n/a, Model=Llama2-13b-chat2023.10 | 26.7 | |
| vanillaCompilation=none, Training=n/a, Model=GPT-3.52023.10 | 24 | |
| vanillaCompilation=none, Training=n/a, Model=Llama2-13b-chat2023.10 | 7 | |
| vanillaCompilation=fewshot, Training=trainset, Model=Llama2-13b-chat2023.10 | 4.3 |