Reasoning on GSM8K (test)
89.6EM AccuracyMCTS
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MCTSModel=Qwen-2.5-7B2025.02 | 89.6 | 79.68 | |
| DPTSModel=Qwen-2.5-7B2025.02 | 89.4 | 19.95 | |
| Best-of-NModel=Qwen-2.5-7B2025.02 | 88.2 | 34.89 | |
| BeamModel=Qwen-2.5-7B2025.02 | 86.7 | 36.49 | |
| LLaDASFT=true, Sequence Length=512, 0-shot=true2025.11 | 81.2 | — | |
| LLaDASFT=false, Sequence Length=512, 0-shot=true2025.11 | 79.5 | — | |
| LLMLinguaShot constraint=1 shot, Tokens=446, 1/τ=5x2024.03 | 79.08 | — | |
| LLMLingua-2Shot constraint=1 shot, Tokens=457, 1/τ=5x2024.03 | 79.08 | — | |
| LLMLingua-2-smallShot constraint=1 shot, Tokens=437, 1/τ=5x2024.03 | 78.92 | — | |
| Full-ShotTokens=2,3662024.03 | 78.85 | — | |
| LLMLingua-2Shot constraint=half shot, Tokens=178, 1/τ=14x2024.03 | 77.79 | — | |
| EDITSequence Length=256, 0-shot=true, SFT=true2025.11 | 77.6 | — | |
| LLMLingua-2-smallShot constraint=half shot, Tokens=161, 1/τ=14x2024.03 | 77.48 | — | |
| LLMLinguaShot constraint=half shot, Tokens=171, 1/τ=14x2024.03 | 77.41 | — | |
| LLaDASFT=true, Sequence Length=256, 0-shot=true2025.11 | 77 | — | |
| EDITSequence Length=512, 0-shot=true, SFT=true2025.11 | 76.2 | — | |
| LLaDASFT=false, Sequence Length=256, 0-shot=true2025.11 | 75.8 | — | |
| DPTSModel=Qwen-2.5-1.5B2025.02 | 75.2 | 18.32 | |
| MCTSModel=Qwen-2.5-1.5B2025.02 | 75.1 | 73.28 | |
| BeamModel=Qwen-2.5-1.5B2025.02 | 71.5 | 41.27 | |
| Best-of-NModel=Qwen-2.5-1.5B2025.02 | 70.1 | 33.37 | |
| LLaDASFT=true, Sequence Length=128, 0-shot=true2025.11 | 69.8 | — | |
| MCTSModel=Llama-3-8B2025.02 | 69.5 | 69.74 | |
| BeamModel=Llama-3-8B2025.02 | 68.3 | 34.51 | |
| DPTSModel=Llama-3-8B2025.02 | 68.2 | 17.82 | |
| LLaDASFT=false, Sequence Length=128, 0-shot=true2025.11 | 68.1 | — | |
| DPTSModel=Llama-3-3B2025.02 | 67.8 | 27.74 | |
| Best-of-NModel=Llama-3-8B2025.02 | 67.6 | 33.48 | |
| EDITSequence Length=128, 0-shot=true, SFT=true2025.11 | 67.3 | — | |
| MCTSModel=Llama-3-3B2025.02 | 64 | 57.19 | |
| GreedyCategory=AR, NFE=512, Zero-shot=true2026.06 | 63.3 | — | |
| BeamModel=Llama-3-3B2025.02 | 58.4 | 28.27 | |
| Best-of-NModel=Llama-3-3B2025.02 | 57.1 | 27.27 | |
| Selective-ContextShot constraint=1 shot, Tokens=452, 1/τ=5x2024.03 | 53.98 | — | |
| SampleCategory=AR, NFE=512, Zero-shot=true2026.06 | 53.9 | — | |
| Selective-ContextShot constraint=half shot, Tokens=218, 1/τ=11x2024.03 | 52.99 | — | |
| Zero-ShotTokens=11, 1/τ=215x2024.03 | 48.75 | — | |
| FMLM+Category=Continuous, NFE=32, Zero-shot=true2026.06 | 19 | — | |
| MDLMCategory=Discrete, NFE=1024, Zero-shot=true2026.06 | 18 | — | |
| S-FLMCategory=Continuous, NFE=1024, Zero-shot=true2026.06 | 18 | — | |
| DuoCategory=Discrete, NFE=1024, Zero-shot=true2026.06 | 17.2 | — | |
| FLMCategory=Continuous, NFE=1024, Zero-shot=true2026.06 | 0.3 | — | |
| CANDICategory=Continuous, NFE=1024, Zero-shot=true2026.06 | 0.2 | — |