Logical Reasoning on FLD
75.6AccuracyCRAFT
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CRAFTBackbone=o4-mini2026.04 | 75.6 | 73 | 7.2 | |
| CRAFTBackbone=GPT-5.4-nano2026.04 | 71.6 | 81 | 9.9 | |
| RAPBackbone=o4-mini2026.04 | 70.4 | 72 | 6.5 | |
| Self-AggregationBackbone=GPT-5.4-nano2026.04 | 68.3 | 67 | 17.1 | |
| Self-RefineBackbone=GPT-5.4-nano2026.04 | 63.4 | 59 | 20.5 | |
| Self-Eval Beam SearchBackbone=o4-mini2026.04 | 62 | 70 | 4.1 | |
| Self-ConsistencyBackbone=GPT-5.4-nano2026.04 | 61.8 | 55 | 14.5 | |
| Univ. Self-ConsistencyBackbone=GPT-5.4-nano2026.04 | 61.4 | 59 | 14.6 | |
| RAPBackbone=GPT-5.4-nano2026.04 | 61.4 | 62 | 12.3 | |
| Self-ConsistencyBackbone=o4-mini2026.04 | 60.6 | 58 | 6 | |
| Faithful CoTBackbone=GPT-5.4-nano2026.04 | 56.8 | 52 | 27.3 | |
| Univ. Self-ConsistencyBackbone=o4-mini2026.04 | 56.2 | 65 | 5.8 | |
| Best-of-NBackbone=GPT-5.4-nano2026.04 | 56 | 48 | 13.9 | |
| Tree-of-ThoughtBackbone=o4-mini2026.04 | 52.4 | 48 | 4.3 | |
| Self-Eval Beam SearchBackbone=GPT-5.4-nano2026.04 | 51.8 | 49 | 1.4 | |
| Faithful CoTBackbone=o4-mini2026.04 | 48 | 52 | 8.9 | |
| Self-AggregationBackbone=o4-mini2026.04 | 46.8 | 45 | 7 | |
| Self-RefineBackbone=o4-mini2026.04 | 44.8 | 51 | 9.3 | |
| Best-of-NBackbone=o4-mini2026.04 | 44 | 43 | 4.7 | |
| Tree-of-ThoughtBackbone=GPT-5.4-nano2026.04 | 24 | 28 | 10.5 |