Mathematical Reasoning on GSM8K (test) (PPL, pass@1, Throughput)
1.25Perplexity (PPL)AR Transformer
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| AR Transformer2026.07 | 1.25 | 75.74 | 67.16 | |
| BD3LMS=42026.07 | 1.41 | 63.53 | 55.39 | |
| SW-SetDLMS=≤ 82026.07 | 1.42 | 66.41 | 60.42 | |
| SW-SetDLMS=≤ 162026.07 | 1.5 | 63.91 | 56.5 | |
| BD3LMS=82026.07 | 1.53 | 56.94 | 54.67 | |
| SW-SetDLMS=≤ 322026.07 | 1.59 | 61.94 | 55.08 | |
| BD3LMS=162026.07 | 1.67 | 50.49 | 52.18 | |
| MDLMdecoding_strategy=confidence-aware parallel decoding, threshold=0.9, S=322026.07 | 2.1 | 6.37 | 24.48 |