Arithmetic Reasoning on Countdown 0-shot (test)
71.5Pass@1 (Greedy)SPG w/ EUBO
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| SPG w/ EUBOMode=0-shot, Temperature=0.9, Generation length=256, Checkpoint selection=best2025.10 | 71.5 | 68.2 | 71.9 | 73.9 | 76.6 | |
| SPG w/ mixtureMode=0-shot, Temperature=0.9, Generation length=256, Checkpoint selection=best2025.10 | 71.1 | 67.5 | 72.5 | 75.1 | 76.6 | |
| WD1Mode=0-shot, Temperature=0.9, Generation length=256, Checkpoint selection=best2025.10 | 54.7 | 44.3 | 60.6 | 68 | 73.1 | |
| UniGRPOMode=0-shot, Temperature=0.9, Generation length=256, Checkpoint selection=best2025.10 | 44.9 | 36.8 | 55.2 | 65 | 72.3 | |
| D1Mode=0-shot, Temperature=0.9, Generation length=256, Checkpoint selection=best2025.10 | 32.4 | 24.5 | 40.4 | 51.4 | 60.6 | |
| LLaDA-1.5Mode=0-shot, Temperature=0.9, Generation length=256, Checkpoint selection=best2025.10 | 21.1 | 18.2 | 32.1 | 42.5 | 50 | |
| LLaDA-8B-InstructMode=0-shot, Temperature=0.9, Generation length=256, Checkpoint selection=best2025.10 | 16.8 | 15.8 | 28.1 | 37.7 | 45.3 |