Game of 24 on Game of 24 100 tasks GPT-4
74Success RateToT
Evaluation results
| Method | Links | |
|---|---|---|
| ToTBreadth (b)=52023.05 | 74 | |
| CoT (best of 100)Sample size (k)=100, Selection=best of k2023.05 | 49 | |
| ToTBreadth (b)=12023.05 | 45 | |
| IO (best of 100)Sample size (k)=100, Selection=best of k2023.05 | 33 | |
| IO + RefineSample size (k)=10, Strategy=Iterative Refinement2023.05 | 27 | |
| CoT-SCPrompting strategy=Chain-of-Thought, Sample size (k)=100, Consistency=Self-Consistency2023.05 | 9 | |
| IO promptPrompting strategy=Input-Output2023.05 | 7.3 | |
| CoT promptPrompting strategy=Chain-of-Thought2023.05 | 4 |