Logical Reasoning on Sudoku
94.3AccuracyEGSPO-SA
Evaluation Results
| Method | Links | |
|---|---|---|
| EGSPO-SASeq. Len.=256, n-shot=0-shot2026.03 | 94.3 | |
| EGSPO-SASeq. Len.=Best, n-shot=0-shot2026.03 | 94.3 | |
| SPGSeq. Len.=256, n-shot=3-shot2026.03 | 94 | |
| SPGSeq. Len.=Best, n-shot=3-shot2026.03 | 94 | |
| SPG w/ mixture (ours)Seq Len=256, Evaluation Protocol=3-shot2025.10 | 94 | |
| EGSPO-SASeq. Len.=128, n-shot=0-shot2026.03 | 93.7 | |
| EGSPOSeq. Len.=256, n-shot=0-shot2026.03 | 93.6 | |
| EGSPOSeq. Len.=Best, n-shot=0-shot2026.03 | 93.6 | |
| EGSPO-SASeq. Len.=512, n-shot=0-shot2026.03 | 93.4 | |
| OracleBackbone=LLaDA-8B-Base, Shots=4-shot2026.04 | 93.4 | |
| EGSPOSeq. Len.=128, n-shot=0-shot2026.03 | 93.3 | |
| SPGSeq. Len.=512, n-shot=3-shot2026.03 | 93.1 | |
| SPG w/ mixture (ours)Seq Len=512, Evaluation Protocol=3-shot2025.10 | 93.1 | |
| d2Seq. Len.=Best, n-shot=0-shot2026.03 | 91.9 | |
| OracleBackbone=Dream-7B-Base, Shots=4-shot2026.04 | 91.2 | |
| SPG w/ EUBO (ours)Seq Len=512, Evaluation Protocol=3-shot2025.10 | 89.9 | |
| RL (BF16)Model=Qwen3.5-9B, Bitwidth=BF162026.05 | 89.84 | |
| AISModel=Qwen3.5-9B, Bitwidth=FP82026.05 | 89.45 | |
| Auto-ICLBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.702026.04 | 89.3 | |
| LLaDA-8B-Instruct w/ DAMSeq Len=1282026.02 | 89.2 | |
| EGSPOSeq. Len.=512, n-shot=0-shot2026.03 | 89.1 | |
| FlashRL (TIS)Model=Qwen3.5-9B, Bitwidth=FP82026.05 | 88.67 | |
| LLaDA-8B-Instruct w/ DAMSeq Len=2562026.02 | 88.12 | |
| SPG w/ EUBO (ours)Seq Len=256, Evaluation Protocol=3-shot2025.10 | 87.1 | |
| PrefixBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.682026.04 | 86.4 | |
| RL (FP8 Rollout)Model=Qwen3.5-9B, Bitwidth=FP82026.05 | 85.94 | |
| Auto-ICLBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.752026.04 | 84.4 | |
| PrefixBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.772026.04 | 83 | |
| SPGSeq. Len.=128, n-shot=3-shot2026.03 | 82.9 | |
| SPG w/ mixture (ours)Seq Len=128, Evaluation Protocol=3-shot2025.10 | 82.9 | |
| SPG w/ EUBO (ours)Seq Len=128, Evaluation Protocol=3-shot2025.10 | 81.2 | |
| VanillaBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.762026.04 | 79 | |
| wd1Seq. Len.=256, n-shot=0-shot2026.03 | 76.4 | |
| wd1Seq. Len.=Best, n-shot=0-shot2026.03 | 76.4 | |
| + PlannerSampling Method=+ Planner, Timestep Budget (T)=1282026.04 | 75.2 | |
| BothSampling Method=Both, Timestep Budget (T)=1282026.04 | 74.3 | |
| + EOS Temp.Sampling Method=+ EOS Temp., Timestep Budget (T)=1282026.04 | 74.2 | |
| + EOS Temp.Sampling Method=+ EOS Temp., Timestep Budget (T)=642026.04 | 73.9 | |
| Prob MarginSampling Method=Prob Margin, Timestep Budget (T)=1282026.04 | 73.8 | |
| Prob MarginSampling Method=Prob Margin, Timestep Budget (T)=642026.04 | 71.3 | |
| Top1 ProbSampling Method=Top1 Prob, Timestep Budget (T)=1282026.04 | 70.5 | |
| + PlannerSampling Method=+ Planner, Timestep Budget (T)=642026.04 | 70.5 | |
| TemperatureSampling Method=Temperature, Timestep Budget (T)=1282026.04 | 70.4 | |
| BothSampling Method=Both, Timestep Budget (T)=642026.04 | 69.9 | |
| TemperatureSampling Method=Temperature, Timestep Budget (T)=642026.04 | 69.7 | |
| Top1 ProbSampling Method=Top1 Prob, Timestep Budget (T)=642026.04 | 68.4 | |
| UniGRPOSeq Len=256, Evaluation Protocol=3-shot2025.10 | 67 | |
| Init. PositionSampling Method=Init. Position, Timestep Budget (T)=1282026.04 | 66.9 | |
| RandomBackbone=LLaDA-8B-Base, Shots=4-shot, Latency (s)=0.772026.04 | 63.1 | |
| RandomBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.692026.04 | 63.1 | |
| UniGRPOSeq Len=512, Evaluation Protocol=3-shot2025.10 | 62.9 | |
| wd1Seq. Len.=512, n-shot=0-shot2026.03 | 62.8 | |
| UniGRPOSeq Len=128, Evaluation Protocol=3-shot2025.10 | 59 | |
| Init. PositionSampling Method=Init. Position, Timestep Budget (T)=642026.04 | 56.4 | |
| AISModel=Qwen3-8B, Bitwidth=FP82026.05 | 39.1 | |
| RL (BF16)Model=Qwen3-8B, Bitwidth=BF162026.05 | 38.28 | |
| FlashRL (TIS)Model=Qwen3-8B, Bitwidth=FP82026.05 | 37.5 | |
| RL (FP8 Rollout)Model=Qwen3-8B, Bitwidth=FP82026.05 | 36.33 | |
| WD1 (reproduced)Seq Len=128, Evaluation Protocol=3-shot2025.10 | 33.1 | |
| DMPO-LLaDASequence Length=1282025.10 | 32.76 | |
| D1 (reproduced)Seq Len=256, Evaluation Protocol=3-shot2025.10 | 32.5 | |
| WD1 (reproduced)Seq Len=256, Evaluation Protocol=3-shot2025.10 | 32.1 | |
| VanillaBackbone=Dream-7B-Base, Shots=4-shot, Latency (s)=0.672026.04 | 30 | |
| D1 (reproduced)Seq Len=512, Evaluation Protocol=3-shot2025.10 | 29.3 | |
| LLaDA-1.5Seq Len=512, Evaluation Protocol=3-shot2025.10 | 29 | |
| LLaDA-8B-InstructSeq Len=256, Evaluation Protocol=3-shot2025.10 | 27.7 | |
| LLaDA-1.5Seq Len=256, Evaluation Protocol=3-shot2025.10 | 26.9 | |
| LLaDA-8B-InstructSeq Len=512, Evaluation Protocol=3-shot2025.10 | 26.2 | |
| DMPO-LLaDA-SFTSequence Length=2562025.10 | 25.73 | |
| DMPO-LLaDA-1.5Sequence Length=1282025.10 | 25.34 | |
| DMPO-LLaDA-SFTSequence Length=1282025.10 | 25.2 | |
| DMPO-LLaDASequence Length=2562025.10 | 24.56 | |
| DMPO-LLaDA-1.5Sequence Length=2562025.10 | 24.51 | |
| cGRPO-LLaDASequence Length=1282025.10 | 24.17 | |
| cGRPO-LLaDASequence Length=2562025.10 | 24.17 | |
| LLaDA-8B-Instruct w/ D1Seq Len=1282026.02 | 23.82 | |
| DMPO-LLaDA-SFTSequence Length=5122025.10 | 23.78 | |
| DMPO-LLaDA-1.5Sequence Length=5122025.10 | 23.34 | |
| WD1 (reproduced)Seq Len=512, Evaluation Protocol=3-shot2025.10 | 22.5 | |
| LIFT2Backbone=Dream-7B, H=22026.05 | 22.5 | |
| OursSeq Len=1282025.10 | 22.4 | |
| d1Seq. Len.=128, n-shot=0-shot2026.03 | 22.1 | |
| d1Seq. Len.=Best, n-shot=0-shot2026.03 | 22.1 | |
| d1-LLaDASequence Length=1282025.10 | 21.97 | |
| cGRPO-LLaDASequence Length=5122025.10 | 21.97 | |
| OursSeq Len=2562025.10 | 20.3 | |
| DMPO-LLaDASequence Length=5122025.10 | 19.97 | |
| AncestralSampling Method=Ancestral, Timestep Budget (T)=642026.04 | 19.1 | |
| AncestralSampling Method=Ancestral, Timestep Budget (T)=1282026.04 | 18.5 | |
| diffu-GRPOSeq Len=1282025.10 | 18.4 | |
| LIFT3Backbone=Dream-7B, H=32026.05 | 17.5 | |
| CARTBackbone=Dream-7B2026.05 | 17.2 | |
| d1Seq. Len.=256, n-shot=0-shot2026.03 | 16.7 | |
| Dream-Instruct (7B)Sequence Length=2562025.10 | 16.41 | |
| OursSeq Len=5122025.10 | 16.1 | |
| GIFTBackbone=Dream-7B2026.05 | 16 | |
| WINOSeq Len=2562025.10 | 15.2 | |
| LLaDA-1.5 (8B)Sequence Length=1282025.10 | 15.19 | |
| VanillaBackbone=Dream-7B2026.05 | 14.8 | |
| Dream-Instruct (7B)Sequence Length=1282025.10 | 14.45 |