Planning and Reasoning on Countdown (accuracy)
81.6AccuracyDTM
Evaluation results
| Method | Links | |
|---|---|---|
| DTMSeq Len=256, Shot count=0-SHOT2026.04 | 81.6 | |
| DTMSeq Len=512, Shot count=0-SHOT2026.04 | 76.6 | |
| SPGSeq Len=256, Shot count=0-SHOT2026.04 | 70.7 | |
| SPGSeq Len=512, Shot count=0-SHOT2026.04 | 70.3 | |
| UNIGRPOSeq Len=512, Shot count=0-SHOT2026.04 | 57 | |
| WD1Seq Len=256, Shot count=0-SHOT2026.04 | 52.3 | |
| WD1Seq Len=512, Shot count=0-SHOT2026.04 | 50.8 | |
| UNIGRPOSeq Len=256, Shot count=0-SHOT2026.04 | 43 | |
| D1Seq Len=512, Shot count=0-SHOT2026.04 | 34.4 | |
| D1Seq Len=256, Shot count=0-SHOT2026.04 | 30.9 | |
| LLADA-1.5Seq Len=512, Shot count=0-SHOT2026.04 | 21.5 | |
| LLADA-1.5Seq Len=256, Shot count=0-SHOT2026.04 | 21.1 | |
| LLADA-8B-INST.Seq Len=256, Shot count=0-SHOT2026.04 | 16.8 | |
| LLADA-8B-INST.Seq Len=512, Shot count=0-SHOT2026.04 | 16.8 |