Arithmetic Planning on Countdown
65.6AccuracyPAPO
Evaluation results
| Method | Links | |
|---|---|---|
| PAPOSeq Len=256, SFT=false2026.06 | 65.6 | |
| PAPOSeq Len=512, SFT=false2026.06 | 65.2 | |
| SAPOSeq Len=512, SFT=false2026.06 | 56.3 | |
| SFT + TSE-RewardSeq Len=512, SFT=true2026.06 | 54.7 | |
| UniGRPO†Seq Len=256, SFT=false2026.06 | 53.9 | |
| SAPOSeq Len=256, SFT=false2026.06 | 52 | |
| PAPOSeq Len=128, SFT=false2026.06 | 52 | |
| UniGRPO†Seq Len=128, SFT=false2026.06 | 51.6 | |
| SAPOSeq Len=128, SFT=false2026.06 | 51.6 | |
| UniGRPO†Seq Len=512, SFT=false2026.06 | 43.8 | |
| SFT + TSE-RewardSeq Len=256, SFT=true2026.06 | 42.6 | |
| SFT + diffu-GRPOSeq Len=512, SFT=true2026.06 | 42.2 | |
| SFT + TSE-RewardSeq Len=128, SFT=true2026.06 | 41.5 | |
| diffu-GRPOSeq Len=512, SFT=false2026.06 | 37.1 | |
| SFT + diffu-GRPOSeq Len=128, SFT=true2026.06 | 34.8 | |
| diffu-GRPOSeq Len=128, SFT=false2026.06 | 33.2 | |
| WINOSeq Len=256, SFT=false2026.06 | 33.2 | |
| SFT + diffu-GRPOSeq Len=256, SFT=true2026.06 | 32 | |
| diffu-GRPOSeq Len=256, SFT=false2026.06 | 31.3 | |
| TSE-VoteSeq Len=128, SFT=false2026.06 | 25 | |
| SFTSeq Len=512, SFT=true2026.06 | 23.8 | |
| TSE-VoteSeq Len=256, SFT=false2026.06 | 23.4 | |
| LLaDA-8B-InstructSeq Len=128, SFT=false2026.06 | 20.7 | |
| SFTSeq Len=128, SFT=true2026.06 | 20.3 | |
| LLaDA-8B-InstructSeq Len=256, SFT=false2026.06 | 19.5 | |
| TSE-VoteSeq Len=512, SFT=false2026.06 | 16.4 | |
| LLaDA-8B-InstructSeq Len=512, SFT=false2026.06 | 16 | |
| SFTSeq Len=256, SFT=true2026.06 | 14.5 |