Logical Reasoning on RP LP Domain, Npred ≤ 30, δ ≤ 6
100Accuracydirect w. L = 128
Evaluation Results
| Method | Links | |
|---|---|---|
| direct w. L = 128Reasoning Strategy=Direct, Number of Layers=1282026.05 | 100 | |
| cot w. L = 64Reasoning Strategy=CoT, Number of Layers=642026.05 | 100 | |
| direct w. L = 24Reasoning Strategy=Direct, Number of Layers=242026.05 | 100 | |
| direct w. L = 128Reasoning Strategy=Direct, Number of Layers=1282026.05 | 100 | |
| cot w. L = 64Reasoning Strategy=CoT, Number of Layers=642026.05 | 100 | |
| cot w. L = 128Reasoning Strategy=CoT, Number of Layers=1282026.05 | 100 | |
| direct w. L = 48Reasoning Strategy=Direct, Number of Layers=482026.05 | 99.9 | |
| direct w. L = 64Reasoning Strategy=Direct, Number of Layers=642026.05 | 99.9 | |
| cot w. L = 48Reasoning Strategy=CoT, Number of Layers=482026.05 | 99.9 | |
| cot w. L = 96Reasoning Strategy=CoT, Number of Layers=962026.05 | 99.9 | |
| cot w. L = 128Reasoning Strategy=CoT, Number of Layers=1282026.05 | 99.9 | |
| direct w. L = 96Reasoning Strategy=Direct, Number of Layers=962026.05 | 99.8 | |
| cot w. L = 32Reasoning Strategy=CoT, Number of Layers=322026.05 | 99.8 | |
| cot (baseline)Reasoning Strategy=CoT2026.05 | 99.7 | |
| direct w. L = 32Reasoning Strategy=Direct, Number of Layers=322026.05 | 99.6 | |
| cot w. L = 24Reasoning Strategy=CoT, Number of Layers=242026.05 | 99.6 | |
| direct w. L = 24Reasoning Strategy=Direct, Number of Layers=242026.05 | 99.3 | |
| cot w. L = 16Reasoning Strategy=CoT, Number of Layers=162026.05 | 98.7 | |
| cot (baseline)Reasoning Strategy=CoT2026.05 | 97.4 | |
| direct (baseline)Reasoning Strategy=Direct2026.05 | 96.9 | |
| direct w. L = 16Reasoning Strategy=Direct, Number of Layers=162026.05 | 96.6 | |
| cot w. Nepochs = 60Reasoning Strategy=CoT, N_epochs=602026.05 | 96.5 | |
| cot w. Nepochs = 30Reasoning Strategy=CoT, N_epochs=302026.05 | 96 | |
| direct w. Nepochs = 60Reasoning Strategy=Direct, N_epochs=602026.05 | 87.2 | |
| direct (baseline)Reasoning Strategy=Direct2026.05 | 85.7 | |
| direct w. Nepochs = 30Reasoning Strategy=Direct, N_epochs=302026.05 | 84 |