Mathematical Reasoning on Countdown
85AccuracyRandOpt
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| RandOptModel=OLMo3-7B-Inst2026.03 | 85 | — | — | |
| EGSPO-SASeq. Len.=1282026.03 | 78.5 | — | — | |
| EGSPO-SASeq. Len.=Best2026.03 | 78.5 | — | — | |
| EGSPO-SASeq. Len.=2562026.03 | 77.3 | — | — | |
| EGSPO-SASeq. Len.=5122026.03 | 76.5 | — | — | |
| EGSPOSeq. Len.=5122026.03 | 75.8 | — | — | |
| EGSPOSeq. Len.=Best2026.03 | 75.8 | — | — | |
| ES + TT-MVModel=OLMo3-7B-Inst2026.03 | 75.6 | — | — | |
| SPGSeq. Len.=2562026.03 | 71.5 | — | — | |
| SPGSeq. Len.=Best2026.03 | 71.5 | — | — | |
| EGSPOSeq. Len.=2562026.03 | 71.5 | — | — | |
| SPG w/ EUBO (ours)Seq Len=256, Evaluation Protocol=0-shot2025.10 | 71.5 | — | — | |
| ESModel=OLMo3-7B-Inst2026.03 | 71 | — | — | |
| SPG w/ mixture (ours)Seq Len=256, Evaluation Protocol=0-shot2025.10 | 70.7 | — | — | |
| SPGSeq. Len.=5122026.03 | 70.3 | — | — | |
| EGSPOSeq. Len.=1282026.03 | 70.3 | — | — | |
| SPG w/ mixture (ours)Seq Len=512, Evaluation Protocol=0-shot2025.10 | 70.3 | — | — | |
| PPOModel=OLMo3-7B-Inst2026.03 | 69 | — | — | |
| SPGSeq. Len.=1282026.03 | 68.8 | — | — | |
| SPG w/ mixture (ours)Seq Len=128, Evaluation Protocol=0-shot2025.10 | 68.8 | — | — | |
| GRPOModel=OLMo3-7B-Inst2026.03 | 68.5 | — | — | |
| SPG w/ EUBO (ours)Seq Len=128, Evaluation Protocol=0-shot2025.10 | 68.4 | — | — | |
| SPG w/ EUBO (ours)Seq Len=512, Evaluation Protocol=0-shot2025.10 | 68 | — | — | |
| Best-of-N‡Model=OLMo3-7B-Inst2026.03 | 67.5 | — | — | |
| TT-MV†Model=OLMo3-7B-Inst2026.03 | 66.8 | — | — | |
| ElasticModel=LLaDA2.0-mini2026.05 | 65.62 | — | — | |
| BaseModel=OLMo3-7B-Inst2026.03 | 64.8 | — | — | |
| RandOptModel=Llama3.1-8B-Inst2026.03 | 63.6 | — | — | |
| ES + TT-MVModel=Llama3.1-8B-Inst2026.03 | 62.5 | — | — | |
| StableDRLGeneration Length=5122026.05 | 62.11 | — | — | |
| ES + TT-MVModel=Qwen2.5-3B-Inst2026.03 | 61.9 | — | — | |
| ESModel=Llama3.1-8B-Inst2026.03 | 60.3 | — | — | |
| LLaDA-8B-Instruct w/ DAMSeq Len=1282026.02 | 60.16 | — | — | |
| StableDRLGeneration Length=2562026.05 | 58.98 | — | — | |
| RandOptModel=Qwen2.5-3B-Inst2026.03 | 58.4 | — | — | |
| UniGRPOSeq Len=512, Evaluation Protocol=0-shot2025.10 | 57 | — | — | |
| ESPOGeneration Length=2562026.05 | 56.64 | — | — | |
| d2Seq. Len.=Best2026.03 | 56.6 | — | — | |
| OursSeq Len=5122025.10 | 56.3 | — | — | |
| ESModel=Qwen2.5-3B-Inst2026.03 | 55.6 | — | — | |
| LLaDA-8B-Instruct w/ DAMSeq Len=2562026.02 | 55.47 | — | — | |
| RandOptModel=Qwen2.5-1.5B-Inst2026.03 | 52.7 | — | — | |
| ESPOGeneration Length=5122026.05 | 52.34 | — | — | |
| WD1 (reproduced)Seq Len=256, Evaluation Protocol=0-shot2025.10 | 52.3 | — | — | |
| OursSeq Len=2562025.10 | 52 | — | — | |
| OursSeq Len=1282025.10 | 51.6 | — | — | |
| wd1Seq. Len.=2562026.03 | 51.2 | — | — | |
| wd1Seq. Len.=Best2026.03 | 51.2 | — | — | |
| WD1 (reported)Seq Len=256, Evaluation Protocol=0-shot2025.10 | 51.2 | — | — | |
| WD1 (reproduced)Seq Len=512, Evaluation Protocol=0-shot2025.10 | 50.8 | — | — | |
| WD1 (reproduced)Seq Len=128, Evaluation Protocol=0-shot2025.10 | 48.8 | — | — | |
| SPGGeneration Length=2562026.05 | 47.27 | — | — | |
| wd1Seq. Len.=5122026.03 | 46.1 | — | — | |
| WD1 (reported)Seq Len=512, Evaluation Protocol=0-shot2025.10 | 46.1 | — | — | |
| SPGGeneration Length=5122026.05 | 44.53 | — | — | |
| UniGRPOSeq Len=128, Evaluation Protocol=0-shot2025.10 | 44.5 | — | — | |
| ESModel=Qwen2.5-1.5B-Inst2026.03 | 44.2 | — | — | |
| LLaDA-8B-Instruct w/ D1Seq Len=1282026.02 | 43.75 | — | — | |
| UniGRPOSeq Len=256, Evaluation Protocol=0-shot2025.10 | 43 | — | — | |
| d1Seq. Len.=5122026.03 | 42.2 | — | — | |
| d1Seq. Len.=Best2026.03 | 42.2 | — | — | |
| d1-LLaDA (reported)Seq Len=512, Evaluation Protocol=0-shot2025.10 | 42.2 | — | — | |
| BaselineModel=LLaDA2.0-mini2026.05 | 41.41 | — | — | |
| Best-of-N‡Model=Llama3.1-8B-Inst2026.03 | 40 | — | — | |
| ES + TT-MVModel=Qwen2.5-1.5B-Inst2026.03 | 39.7 | — | — | |
| wd1Generation Length=2562026.05 | 39.45 | — | — | |
| wd1Generation Length=5122026.05 | 38.67 | — | — | |
| Elastic-dLLMBase Model=LLaDA-8B-Instruct2026.05 | 38.67 | — | — | |
| diffu-GRPOSeq Len=5122025.10 | 37.1 | — | — | |
| D1 (reported)Seq Len=512, Evaluation Protocol=0-shot2025.10 | 37.1 | — | — | |
| PPOModel=Qwen2.5-3B-Inst2026.03 | 35.3 | — | — | |
| d1Seq. Len.=1282026.03 | 34.8 | — | — | |
| d1-LLaDA (reported)Seq Len=128, Evaluation Protocol=0-shot2025.10 | 34.8 | — | — | |
| LLaDA1.5-8BDecoding Strategy=FourierSampler2026.01 | 34.77 | — | — | |
| LLaDA-8B-InstructSeq Len=1282026.02 | 34.77 | — | — | |
| Order-Token SearchBackbone=LLaDA, Seq Len=1282026.01 | 34.4 | — | — | |
| D1 (reproduced)Seq Len=512, Evaluation Protocol=0-shot2025.10 | 34.4 | — | — | |
| diffu-GRPOSeq Len=1282025.10 | 33.2 | — | — | |
| WINOSeq Len=2562025.10 | 33.2 | — | — | |
| D1 (reported)Seq Len=128, Evaluation Protocol=0-shot2025.10 | 33.2 | — | — | |
| GRPOModel=Qwen2.5-3B-Inst2026.03 | 32.6 | — | — | |
| d1Seq. Len.=2562026.03 | 32 | — | — | |
| d1-LLaDA (reported)Seq Len=256, Evaluation Protocol=0-shot2025.10 | 32 | — | — | |
| Order-Token SearchBackbone=LLaDA-1.5, Seq Len=1282026.01 | 31.3 | — | — | |
| LLaDA-8B-Instruct w/ D1Seq Len=2562026.02 | 31.3 | — | — | |
| diffu-GRPOSeq Len=2562025.10 | 31.3 | — | — | |
| D1 (reported)Seq Len=256, Evaluation Protocol=0-shot2025.10 | 31.3 | — | — | |
| D1 (reproduced)Seq Len=128, Evaluation Protocol=0-shot2025.10 | 30.9 | — | — | |
| D1 (reproduced)Seq Len=256, Evaluation Protocol=0-shot2025.10 | 30.9 | — | — | |
| TT-MV†Model=Qwen2.5-1.5B-Inst2026.03 | 30.8 | — | — | |
| LLaDA1.5-8BDecoding Strategy=Vanilla2026.01 | 30.47 | — | — | |
| RandOptModel=OLMo3-7B2026.03 | 30.2 | — | — | |
| LLaDA1.5-8BDecoding Strategy=RWS2026.01 | 30.08 | — | — | |
| LLaDA-8B-InstructDecoding Strategy=RWS2026.01 | 30.08 | — | — | |
| Elastic-dLLMBase Model=LLaDA-1.52026.05 | 29.69 | — | — | |
| Iter-SFTBackbone=Qwen2.5-3B-Instruct2026.05 | 29 | — | — | |
| LLaDA-8B-InstructDecoding Strategy=FourierSampler2026.01 | 28.9 | — | — | |
| GRPOModel=OLMo3-7B2026.03 | 28.8 | — | — | |
| Best-of-N‡Model=Qwen2.5-3B-Inst2026.03 | 28.5 | — | — | |
| DiSE-flexibleModel=LLaDA-Instruct-8B, Base Len=1282026.03 | 27.73 | — | — |