Reasoning on Sudoku
96Pass@1BoN w. ReJump
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BoN w. ReJumpModel=Grok 3 Mini Beta2025.11 | 96 | — | — | 0.71 | — | |
| SPG w/ MixtureSeq Len=2562026.03 | 94 | — | — | — | — | |
| SPG w/ MixtureSeq Len=5122026.03 | 93.1 | — | — | — | — | |
| CAPRBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 93 | — | — | — | — | |
| d-TreeRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 92.9 | — | — | — | — | |
| ESPOSeq Len=1282026.03 | 92.7 | — | — | — | — | |
| CAPRBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 92.7 | — | — | — | — | |
| StableDRLSeq Len=2562026.03 | 92.4 | — | — | — | — | |
| StableDRLSeq Len=1282026.03 | 91.9 | — | — | — | — | |
| Majority VoteModel=Grok 3 Mini Beta2025.11 | 91 | — | — | 6.01 | — | |
| StableDRLSeq Len=5122026.03 | 90.1 | — | — | — | — | |
| CAPRBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 84.8 | — | — | — | — | |
| ESPOSeq Len=2562026.03 | 84.7 | — | — | — | — | |
| SPG w/ MixtureSeq Len=1282026.03 | 82.9 | — | — | — | — | |
| CAPRBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 81 | — | — | — | — | |
| ESPOSeq Len=5122026.03 | 80.5 | — | — | — | — | |
| d-TreeRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 80.3 | — | — | — | — | |
| d-TreeRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 78.3 | — | — | — | — | |
| d2-stepMergeBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 76.1 | — | — | — | — | |
| d-TreeRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 72 | — | — | — | — | |
| UniGRPOSeq Len=2562026.03 | 67 | — | — | — | — | |
| d2-stepMergeBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 66.2 | — | — | — | — | |
| UniGRPOSeq Len=5122026.03 | 62.9 | — | — | — | — | |
| UniGRPOSeq Len=1282026.03 | 59 | — | — | — | — | |
| wd1Backbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 40.6 | — | — | — | — | |
| wd1Backbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 35.4 | — | — | — | — | |
| WD1Seq Len=1282026.03 | 33.1 | — | — | — | — | |
| D1Seq Len=2562026.03 | 32.5 | — | — | — | — | |
| WD1Seq Len=2562026.03 | 32.1 | — | — | — | — | |
| GDPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 31.2 | — | — | — | — | |
| GDPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 30.6 | — | — | — | — | |
| D1Seq Len=5122026.03 | 29.3 | — | — | — | — | |
| LLaDA-1.5Seq Len=5122026.03 | 29 | — | — | — | — | |
| LLaDA-8B-Inst.Seq Len=2562026.03 | 27.7 | — | — | — | — | |
| TraceRLBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 27.4 | — | — | — | — | |
| LLaDA-1.5Seq Len=2562026.03 | 26.9 | — | — | — | — | |
| LLaDA-8B-Inst.Seq Len=5122026.03 | 26.2 | — | — | — | — | |
| TraceRLBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 25.6 | — | — | — | — | |
| TraceRLBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 25.4 | — | — | — | — | |
| wd1Backbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 25.2 | — | — | — | — | |
| TraceRLBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 25.2 | — | — | — | — | |
| wd1Backbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 24.2 | — | — | — | — | |
| WD1Seq Len=5122026.03 | 22.5 | — | — | — | — | |
| SAPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 21.2 | — | — | — | — | |
| SAPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 20.3 | — | — | — | — | |
| SAPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 18.7 | — | — | — | — | |
| Diffu-GRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 17.3 | — | — | — | — | |
| SAPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 16.1 | — | — | — | — | |
| Diffu-GRPOBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 15.4 | — | — | — | — | |
| Diffu-GRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 12.9 | — | — | — | — | |
| VRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 12.8 | — | — | — | — | |
| BaseBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 12.7 | — | — | — | — | |
| Diffu-GRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 11.2 | — | — | — | — | |
| VRPOBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 9.6 | — | — | — | — | |
| BaseBackbone=LLaDA-MoE-7B-A1B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 8.1 | — | — | — | — | |
| LLaDA-1.5Seq Len=1282026.03 | 7.4 | — | — | — | — | |
| D1Seq Len=1282026.03 | 7.2 | — | — | — | — | |
| BaseBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=256, Zero-shot setting=true2026.06 | 6.7 | — | — | — | — | |
| LLaDA-8B-Inst.Seq Len=1282026.03 | 5.7 | — | — | — | — | |
| BaseBackbone=LLaDA-8B-Instruct, Decoding Budget (L)=512, Zero-shot setting=true2026.06 | 5.5 | — | — | — | — | |
| Baseline (Full Steps)Sequence Length=1282025.11 | — | 64 | — | — | — | |
| Baseline (Full Steps)Sequence Length=2562025.11 | — | 128 | — | — | — | |
| Baseline (Full Steps)Sequence Length=5122025.11 | — | 256 | — | — | — | |
| D1Seq Len=Average2026.03 | — | — | — | — | 23 | |
| EDITSequence Length=1282025.11 | — | 38.3 | 40.2 | — | — | |
| EDITSequence Length=2562025.11 | — | 74.9 | 41.5 | — | — | |
| EDITSequence Length=5122025.11 | — | 163.3 | 36.2 | — | — | |
| ESPOSeq Len=Average2026.03 | — | — | — | — | 86 | |
| LLaDA-1.5Seq Len=Average2026.03 | — | — | — | — | 21.1 | |
| LLaDA-8B-Inst.Seq Len=Average2026.03 | — | — | — | — | 19.9 | |
| SPG w/ MixtureSeq Len=Average2026.03 | — | — | — | — | 90 | |
| StableDRLSeq Len=Average2026.03 | — | — | — | — | 91.5 | |
| UniGRPOSeq Len=Average2026.03 | — | — | — | — | 63 | |
| WD1Seq Len=Average2026.03 | — | — | — | — | 29.2 |