Puzzle Solving on Pocket Cube (test)
0.847AccuracyXoT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| XoTBackbone LLM=GPT-3.5, Revisions (r)=32023.11 | 0.847 | 2.01 | 103.22 | |
| XoTBackbone LLM=GPT-4, Revisions (r)=32023.11 | 0.8361 | 2 | 84.63 | |
| XoTBackbone LLM=GPT-3.5, Revisions (r)=22023.11 | 0.8033 | 1.81 | 96.46 | |
| XoTBackbone LLM=GPT-4, Revisions (r)=22023.11 | 0.7932 | 1.79 | 146.52 | |
| XoTBackbone LLM=GPT-4, Revisions (r)=12023.11 | 0.776 | 1.54 | 75.51 | |
| XoTBackbone LLM=GPT-3.5, Revisions (r)=12023.11 | 0.7432 | 1.55 | 64.63 | |
| MCTS2023.11 | 0.4644 | — | — | |
| ToTBackbone LLM=GPT-4, Search Width (b)=32023.11 | 0.1957 | 56.58 | — | |
| GoTBackbone LLM=GPT-4, Top-k (k)=12023.11 | 0.1803 | 8.55 | — | |
| ToTBackbone LLM=GPT-3.5, Search Width (b)=32023.11 | 0.1749 | 58.72 | — | |
| ToTBackbone LLM=GPT-4, Search Width (b)=12023.11 | 0.1148 | 16.39 | — | |
| ToTBackbone LLM=GPT-3.5, Search Width (b)=12023.11 | 0.0765 | 16.5 | — | |
| GoTBackbone LLM=GPT-3.5, Top-k (k)=12023.11 | 0.0164 | 8.93 | — | |
| IOBackbone LLM=GPT-3.52023.11 | 0.0109 | 1 | — | |
| IOBackbone LLM=GPT-42023.11 | 0.0109 | 1 | — | |
| CoTBackbone LLM=GPT-42023.11 | 0.0109 | 1 | — | |
| CoT-SCBackbone LLM=GPT-42023.11 | 0.0109 | 10 | — | |
| CoTBackbone LLM=GPT-3.52023.11 | 0 | 1 | — | |
| CoT-SCBackbone LLM=GPT-3.52023.11 | 0 | 10 | — | |
| LLaMA-2-13B2023.11 | 0 | — | — |