Multi-Solution Arithmetic Reasoning on Game of 24 (4nu 137 test)
76.25Multi Solution AccuracyXoT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| XoTLLM Backbone=GPT-4, Number of thought revisions (r)=12023.11 | 76.25 | 2.36 | 2.31 | 109.64 | |
| XoTLLM Backbone=GPT-3.5, Number of thought revisions (r)=12023.11 | 62.9 | 2.29 | 3.51 | 116.34 | |
| ToTLLM Backbone=GPT-4, Branching factor (b)=32023.11 | 39.9 | 2.78 | 39.83 | — | |
| GoTLLM Backbone=GPT-4, k-value (k)=32023.11 | 10.46 | 1.39 | 7 | — | |
| IOLLM Backbone=GPT-42023.11 | 8.27 | 2.99 | 1 | — | |
| GoTLLM Backbone=GPT-3.5, k-value (k)=32023.11 | 8.03 | 1.93 | 7 | — | |
| CoT-SCLLM Backbone=GPT-42023.11 | 8.03 | 2.99 | 10 | — | |
| CoTLLM Backbone=GPT-42023.11 | 7.79 | 2.94 | 1 | — | |
| IOLLM Backbone=GPT-3.52023.11 | 4.87 | 2.88 | 1 | — | |
| ToTLLM Backbone=GPT-3.5, Branching factor (b)=32023.11 | 3.41 | 2.99 | 43.96 | — | |
| CoT-SCLLM Backbone=GPT-3.52023.11 | 1.7 | 2.76 | 10 | — | |
| CoTLLM Backbone=GPT-3.52023.11 | 1.22 | 2.77 | 1 | — |