Constraint Satisfaction Reasoning on ZebraLogic
96.8Easy ScoreGPT-4o
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPT-4oPrompting=decl. PL-1S2025.12 | 96.8 | 44 | 58.8 | |
| GPT-4oPrompting=decl. PY-1S2025.12 | 96.4 | 62.1 | 71.7 | |
| Claude 3.5 SonnetPrompting=CoT2025.12 | 87.5 | 12.4 | 33.4 | |
| Llama-3.1-405BPrompting=CoT2025.12 | 87.1 | 11.4 | 32.6 | |
| GPT-4oPrompting=CoT2025.12 | 77.9 | 8.9 | 28.2 | |
| CodeLlama13BPrompting=decl. PY-1S2025.12 | 74.3 | 20 | 35.2 | |
| GPT-4oPrompting=PY-ZS2025.12 | 69.7 | 26.5 | 38.1 | |
| GPT-4oPrompting=PL-ZS2025.12 | 64.6 | 15 | 28.9 | |
| CodeLlama13BPrompting=decl. PL-1S2025.12 | 62.5 | 14.6 | 28.5 |