Planning on Countdown
87.5AccuracyBGPO
Evaluation Results
| Method | Links | |
|---|---|---|
| BGPOBase Model=LLaDA-8B-Instruct, RL Algorithm=BGPO2025.10 | 87.5 | |
| VRPO-OLBase Model=LLaDA-8B-Instruct, RL Algorithm=VRPO-OL, Sampling size (nt)=42025.10 | 84.8 | |
| BGPO (off-policy)Base Model=LLaDA-8B-Instruct, RL Algorithm=BGPO, Mode=off-policy2025.10 | 84.8 | |
| ESPOBackbone=LLaDA-8B, Seq Len=2562025.12 | 82 | |
| ESPOBackbone=LLaDA-8B, Seq Len=1282025.12 | 81.6 | |
| ESPOBackbone=LLaDA-8B, Seq Len=Avg.2025.12 | 81 | |
| ESPOBackbone=LLaDA-8B, Seq Len=5122025.12 | 79.3 | |
| RSPOGeneration length=2562026.05 | 78.83 | |
| RSPOGeneration length=5122026.05 | 73.83 | |
| ESPOBackbone=Dream-7B, Seq Len=1282025.12 | 68.8 | |
| ESPOBackbone=Dream-7B, Seq Len=2562025.12 | 66.8 | |
| ESPOBackbone=Dream-7B, Seq Len=Avg.2025.12 | 66.8 | |
| ESPOBackbone=Dream-7B, Seq Len=5122025.12 | 64.8 | |
| SAPOGeneration length=5122026.05 | 56.3 | |
| diffu-GRPOBase Model=LLaDA-8B-Instruct, RL Algorithm=diffu-GRPO2025.10 | 53.1 | |
| TraceRLGeneration length=5122026.05 | 52.6 | |
| SAPOGeneration length=2562026.05 | 52 | |
| wd1Backbone=LLaDA-8B, Seq Len=2562025.12 | 51.2 | |
| wd1Generation length=2562026.05 | 51.2 | |
| TraceRLGeneration length=2562026.05 | 50.4 | |
| wd1Backbone=LLaDA-8B, Seq Len=Avg.2025.12 | 48.3 | |
| wd1Backbone=LLaDA-8B, Seq Len=1282025.12 | 47.7 | |
| wd1Backbone=LLaDA-8B, Seq Len=5122025.12 | 46.1 | |
| wd1Generation length=5122026.05 | 46.1 | |
| wd12025.10 | 46.1 | |
| Info-Gain SamplerK=1, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 45.2 | |
| Info-Gain SamplerK=2, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 44.1 | |
| PC-SamplerK=2, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 42.4 | |
| PC-Sampler w/ E-BONModel=LLaDA-Instruct-8B, Decoding Strategy=PC-Sampler + E-BON2025.12 | 42.2 | |
| wd1Backbone=Dream-7B, Seq Len=5122025.12 | 42.2 | |
| PC-SamplerK=1, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 41.8 | |
| PC-Sampler w/ E-SMCModel=LLaDA-Instruct-8B, Decoding Strategy=PC-Sampler + E-SMC2025.12 | 40.4 | |
| PC-Sampler w/ E-BONModel=LLaDA-1.5-8B, Decoding Strategy=PC-Sampler + E-BON2025.12 | 40.2 | |
| PC-Sampler w/ E-SMCModel=LLaDA-1.5-8B, Decoding Strategy=PC-Sampler + E-SMC2025.12 | 39.3 | |
| ConfidenceK=1, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 39.2 | |
| wd1Backbone=Dream-7B, Seq Len=2562025.12 | 37.9 | |
| d-OPSDSeq Len=1282026.06 | 37.9 | |
| diffu-GRPOBackbone=Dream-7B, Seq Len=5122025.12 | 37.5 | |
| diffu-GRPOBackbone=LLaDA-8B, Seq Len=5122025.12 | 37.1 | |
| d1Generation length=5122026.05 | 37.1 | |
| d3LLMSeq Len=1282026.06 | 36.7 | |
| PC-SamplerModel=LLaDA-Instruct-8B, Decoding Strategy=PC-Sampler2025.12 | 36.3 | |
| wd1Backbone=Dream-7B, Seq Len=Avg.2025.12 | 36.3 | |
| ConfidenceK=2, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 36.2 | |
| KLASSK=2, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 35.4 | |
| MarginK=1, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 35.2 | |
| PC-SamplerModel=LLaDA-1.5-8B, Decoding Strategy=PC-Sampler2025.12 | 35 | |
| EntropyModel=LLaDA-1.5-8B, Decoding Strategy=Entropy2025.12 | 34.7 | |
| EB-SamplerModel=LLaDA-1.5-8B, Decoding Strategy=EB-Sampler2025.12 | 34.6 | |
| KLASSK=1, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 34.3 | |
| P2Model=LLaDA-Instruct-8B, Decoding Strategy=P22025.12 | 34.2 | |
| EB-SamplerModel=LLaDA-Instruct-8B, Decoding Strategy=EB-Sampler2025.12 | 34.1 | |
| ConfidenceModel=LLaDA-Instruct-8B, Decoding Strategy=Confidence2025.12 | 34 | |
| MarginModel=LLaDA-Instruct-8B, Decoding Strategy=Margin2025.12 | 33.9 | |
| diffu-GRPOBackbone=LLaDA-8B, Seq Len=Avg.2025.12 | 33.9 | |
| EntropyModel=LLaDA-Instruct-8B, Decoding Strategy=Entropy2025.12 | 33.8 | |
| ConfidenceModel=LLaDA-1.5-8B, Decoding Strategy=Confidence2025.12 | 33.8 | |
| P2Model=LLaDA-1.5-8B, Decoding Strategy=P22025.12 | 33.6 | |
| EntropyK=1, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 33.6 | |
| diffu-GRPOBackbone=LLaDA-8B, Seq Len=1282025.12 | 33.2 | |
| diffu-GRPOSeq Len=1282026.06 | 33.2 | |
| Fast-dLLMModel=LLaDA-1.5-8B, Decoding Strategy=Fast-dLLM2025.12 | 32.9 | |
| Semi-ARModel=LLaDA-Instruct-8B, Decoding Strategy=Semi-AR2025.12 | 32.6 | |
| Semi-ARModel=LLaDA-1.5-8B, Decoding Strategy=Semi-AR2025.12 | 32.4 | |
| d-OPSDSeq Len=2562026.06 | 32.3 | |
| d1-LLaDA2025.10 | 32 | |
| MarginModel=LLaDA-1.5-8B, Decoding Strategy=Margin2025.12 | 31.8 | |
| diffu-GRPOBackbone=LLaDA-8B, Seq Len=2562025.12 | 31.3 | |
| d1Generation length=2562026.05 | 31.3 | |
| diffu-GRPOSeq Len=2562026.06 | 31.3 | |
| diffu-GRPOBackbone=Dream-7B, Seq Len=Avg.2025.12 | 30.8 | |
| wd1Backbone=Dream-7B, Seq Len=1282025.12 | 28.9 | |
| MarginK=2, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 28.9 | |
| diffu-GRPOBackbone=Dream-7B, Seq Len=2562025.12 | 27.7 | |
| diffu-GRPOBackbone=Dream-7B, Seq Len=1282025.12 | 27.3 | |
| EntropyK=2, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 23.2 | |
| VRPOGeneration length=2562026.05 | 22.3 | |
| VRPOSeq Len=1282026.06 | 21.9 | |
| VRPOSeq Len=2562026.06 | 21.1 | |
| LLaDABackbone=LLaDA-8B, Seq Len=1282025.12 | 20.7 | |
| LLaDA-8B-InstructSeq Len=1282026.06 | 20.3 | |
| SFT VariantSeq Len=1282026.06 | 20.3 | |
| LLaDABackbone=LLaDA-8B, Seq Len=2562025.12 | 19.5 | |
| LLaDA-8B-InstructGeneration length=2562026.05 | 19.5 | |
| LLaDA-8B-InstructBase Model=LLaDA-8B-Instruct2025.10 | 19.5 | |
| LLaDA-8B-InstructSeq Len=2562026.06 | 19.1 | |
| LLaDABackbone=LLaDA-8B, Seq Len=Avg.2025.12 | 18.7 | |
| VRPOGeneration length=5122026.05 | 18 | |
| DreamBackbone=Dream-7B, Seq Len=5122025.12 | 17.4 | |
| LLaDABackbone=LLaDA-8B, Seq Len=5122025.12 | 16 | |
| LLaDA-8B-InstructGeneration length=5122026.05 | 16 | |
| UniformModel=LLaDA-1.5-8B, Decoding Strategy=Uniform2025.12 | 15.8 | |
| SFT VariantSeq Len=2562026.06 | 14.5 | |
| UniformModel=LLaDA-Instruct-8B, Decoding Strategy=Uniform2025.12 | 14.4 | |
| d3LLMSeq Len=2562026.06 | 12.5 | |
| Fast-dLLMModel=LLaDA-Instruct-8B, Decoding Strategy=Fast-dLLM2025.12 | 11.4 | |
| DreamBackbone=Dream-7B, Seq Len=Avg.2025.12 | 11.2 | |
| UniformK=1, Backbone=Dream-7B, Decoding Strategy=Global, Attention Type=full-attention2026.02 | 8.8 | |
| DreamBackbone=Dream-7B, Seq Len=1282025.12 | 8.5 | |
| DreamBackbone=Dream-7B, Seq Len=2562025.12 | 7.8 |