Logical Reasoning on Holiday Puzzle (test)
24.5AccuracyDID
Evaluation results
| Method | Links | |
|---|---|---|
| DIDBase Model=Claude 3.5 Sonnet2024.10 | 24.5 | |
| ToTBase Model=Claude 3.5 Sonnet2024.10 | 24 | |
| CoTBase Model=Claude 3.5 Sonnet2024.10 | 17.8 | |
| IOBase Model=Claude 3.5 Sonnet2024.10 | 17.4 | |
| DIDBase Model=GPT-4o2024.10 | 15.4 | |
| IOBase Model=GPT-4o2024.10 | 7.8 | |
| ToTBase Model=GPT-4o2024.10 | 7.5 | |
| DIDBase Model=GPT-3.5 Turbo2024.10 | 5.6 | |
| CoTBase Model=GPT-4o2024.10 | 5.2 | |
| ToTBase Model=GPT-3.5 Turbo2024.10 | 2 | |
| CoTBase Model=GPT-3.5 Turbo2024.10 | 1.4 | |
| IOBase Model=GPT-3.5 Turbo2024.10 | 0.2 |