Reasoning on COUNTDOWN (test)
66.02AccuracySTP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| STPBase model=LLaDA-8B-Instruct2026.02 | 66.02 | 142,055 | |
| GRPO w/ ELBOBase model=LLaDA-8B-Instruct2026.02 | 36.33 | 164,657 | |
| EDITSequence Length=256, 0-shot=true, SFT=true2025.11 | 31.6 | — | |
| EDITSequence Length=128, 0-shot=true, SFT=true2025.11 | 28.9 | — | |
| EDITSequence Length=512, 0-shot=true, SFT=true2025.11 | 27.7 | — | |
| Diffu-GRPOBase model=LLaDA-8B-Instruct2026.02 | 25.39 | 147,100 | |
| LLaDASFT=true, Sequence Length=256, 0-shot=true2025.11 | 20.7 | — | |
| LLaDASFT=true, Sequence Length=512, 0-shot=true2025.11 | 20.3 | — | |
| LLaDASFT=false, Sequence Length=128, 0-shot=true2025.11 | 19.9 | — | |
| LLaDASFT=false, Sequence Length=256, 0-shot=true2025.11 | 19.5 | — | |
| LLaDASFT=true, Sequence Length=128, 0-shot=true2025.11 | 19.5 | — | |
| LLaDA-8B-InstructBase model=LLaDA-8B-Instruct2026.02 | 16.8 | — | |
| LLaDASFT=false, Sequence Length=512, 0-shot=true2025.11 | 16.4 | — |