Constraint Satisfaction Reasoning on Countdown
72.7Pass@1CAPR
Evaluation results
| Method | Links | |
|---|---|---|
| CAPRBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 72.7 | |
| CAPRBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 72.7 | |
| d-TreeRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 71.1 | |
| CAPRBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 69.9 | |
| CAPRBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 68.8 | |
| d-TreeRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 67.2 | |
| d-TreeRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 62.1 | |
| d-TreeRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 60.6 | |
| wd1Backbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 58.7 | |
| GDPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 58.1 | |
| wd1Backbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 56.6 | |
| SAPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 56.3 | |
| SAPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 54.8 | |
| SAPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 54.2 | |
| TraceRLBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 54.2 | |
| GDPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 53.4 | |
| TraceRLBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 52.6 | |
| d2-stepMergeBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 52.4 | |
| d2-stepMergeBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 52.1 | |
| SAPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 52 | |
| wd1Backbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 51.2 | |
| TraceRLBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 50.4 | |
| Diffu-GRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 50.1 | |
| TraceRLBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 49.1 | |
| Diffu-GRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 48.2 | |
| wd1Backbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 46.1 | |
| BaseBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 42.6 | |
| BaseBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 41.4 | |
| Diffu-GRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 37.1 | |
| Diffu-GRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 31.3 | |
| VRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 22.3 | |
| BaseBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 19.5 | |
| VRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 18 | |
| BaseBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 16 |