Logical Reasoning on Zebra Puzzles 114 (test)
0.761Average PSGPT-4
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4Temperature (T)=Var., Decomposition agent (D)=No2024.07 | 0.761 | 72 | 1.667 | |
| GPT-4Temperature (T)=0, Decomposition agent (D)=Yes2024.07 | 0.7 | 63 | 1.333 | |
| GPT-4Temperature (T)=0, Decomposition agent (D)=No2024.07 | 0.687 | 59 | 1.185 | |
| GPT-3.5Temperature (T)=0, Decomposition agent (D)=Yes2024.07 | 0.52 | 24 | 0.412 | |
| Llama3-8bTemperature (T)=0, Decomposition agent (D)=No2024.07 | 0.496 | 21 | 0.5 | |
| GPT-3.5Temperature (T)=0, Decomposition agent (D)=No2024.07 | 0.493 | 22 | 0.294 | |
| GPT-3.5Temperature (T)=Var., Decomposition agent (D)=No2024.07 | 0.484 | 24 | 0.412 | |
| Llama3-8bTemperature (T)=0, Decomposition agent (D)=Yes2024.07 | 0.468 | 18 | 0.286 | |
| Llama3-8bTemperature (T)=Var., Decomposition agent (D)=No2024.07 | 0.436 | 15 | 0.071 |