Reasoning on Countdown
83.2AccuracyDMPO-LLaDA-1.5
Evaluation Results
| Method | Links | |
|---|---|---|
| DMPO-LLaDA-1.5Sequence Length=5122025.10 | 83.2 | |
| DMPO-LLaDASequence Length=5122025.10 | 82.81 | |
| DMPO-LLaDASequence Length=2562025.10 | 80.86 | |
| DMPO-LLaDA-1.5Sequence Length=2562025.10 | 79.3 | |
| DMPO-LLaDA-SFTSequence Length=5122025.10 | 77.34 | |
| DMPO-LLaDASequence Length=1282025.10 | 67.19 | |
| DMPO-LLaDA-SFTSequence Length=2562025.10 | 67.19 | |
| DMPO-LLaDA-1.5Sequence Length=1282025.10 | 59.77 | |
| StepMerge (d2)Post-training=true2025.09 | 56.6 | |
| DMPO-LLaDA-SFTSequence Length=1282025.10 | 54.69 | |
| wd1Post-training=true, Evaluation Source=Corresponding paper2025.09 | 51.2 | |
| cGRPO-LLaDASequence Length=2562025.10 | 42.58 | |
| diffu-GRPO (d1)Post-training=true, Evaluation Source=Corresponding paper2025.09 | 42.2 | |
| cGRPO-LLaDASequence Length=5122025.10 | 37.11 | |
| d1-LLaDASequence Length=1282025.10 | 34.38 | |
| LIFT2Backbone=LLaDA-1.52026.05 | 31.3 | |
| LIFT3Backbone=LLaDA-1.52026.05 | 31.2 | |
| d1-LLaDASequence Length=5122025.10 | 30.47 | |
| cGRPO-LLaDASequence Length=1282025.10 | 30.08 | |
| Dream-Instruct (7B)Sequence Length=2562025.10 | 28.52 | |
| LIFT2Backbone=LLaDA-8B-Instruct2026.05 | 27.9 | |
| Dream-Instruct (7B)Sequence Length=5122025.10 | 27.34 | |
| AlfaBackbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=69.2M (0.85%), Steps=128, Zero-shot=true2026.03 | 27.3 | |
| d1-LLaDASequence Length=2562025.10 | 26.56 | |
| LIFT3Backbone=LLaDA-8B-Instruct2026.05 | 26.4 | |
| LLaDA-1.5 (8B)Sequence Length=1282025.10 | 26.17 | |
| LLaDA-1.5 (8B)Sequence Length=5122025.10 | 23.83 | |
| LLaDA-Instruct (8B)Sequence Length=1282025.10 | 23.44 | |
| LLaDA 1.5Post-training=false, Evaluation Source=Released checkpoint2025.09 | 23.4 | |
| CARTBackbone=LLaDA-8B-Instruct2026.05 | 23 | |
| Dream-Instruct (7B)Sequence Length=1282025.10 | 22.66 | |
| LLaDA 1.5Backbone=LLaDA-1.52026.05 | 22.6 | |
| VanillaBackbone=LLaDA-1.52026.05 | 22 | |
| GIFTBackbone=LLaDA-8B-Instruct2026.05 | 21.7 | |
| DoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=101.1M (1.25%), Steps=128, Zero-shot=true2026.03 | 21.5 | |
| CARTBackbone=LLaDA-1.52026.05 | 21.5 | |
| VanillaBackbone=LLaDA-8B-Instruct2026.05 | 20.7 | |
| GIFTBackbone=LLaDA-1.52026.05 | 20.7 | |
| LoRABackbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=100.7M (1.24%), Steps=128, Zero-shot=true2026.03 | 20.3 | |
| LLaDAPost-training=false, Evaluation Source=Released checkpoint2025.09 | 19.9 | |
| LLaDABackbone=LLaDA-8B-Instruct2026.05 | 19.6 | |
| AlfaBackbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=69.2M (0.85%), Steps=256, Zero-shot=true2026.03 | 17.2 | |
| LLaDA-1.5 (8B)Sequence Length=2562025.10 | 16.41 | |
| LoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=50.3M (0.62%), Steps=128, Zero-shot=true2026.03 | 16.4 | |
| DoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=101.1M (1.25%), Steps=256, Zero-shot=true2026.03 | 16 | |
| LLaDA-Instruct (8B)Sequence Length=5122025.10 | 14.84 | |
| LoRABackbone=LLaDA-8B-Instruct, LoRA Rank=128, Tuned Params (Usage %)=100.7M (1.24%), Steps=256, Zero-shot=true2026.03 | 14.5 | |
| LLaDA-Instruct (8B)Sequence Length=2562025.10 | 14.45 | |
| LoRA†Backbone=LLaDA-8B-Instruct, LoRA Rank=64, Tuned Params (Usage %)=50.3M (0.62%), Steps=256, Zero-shot=true2026.03 | 12.5 |