Mathematical Reasoning on Math500 (Pass@1, TPS, Latency)
93.5Pass@1 AccTarget Model
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Target ModelDraft Model=DeepseekR1-1.5B, Target Model=QwQ-32B2026.02 | 93.5 | 40.71 | 99.04 | |
| Target ModelDraft Model=DeepseekR1-1.5B, Target Model=DeepseekR1-32B2026.02 | 92.8 | 39.21 | 89.91 | |
| SemanticSpecDraft Model=DeepseekR1-1.5B, Target Model=QwQ-32B2026.02 | 92.17 | 96.95 | 43.53 | |
| SemanticSpecDraft Model=DeepseekR1-1.5B, Target Model=DeepseekR1-32B2026.02 | 91 | 117.36 | 35.47 | |
| PreSafebase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 90.7 | — | — | |
| SafePathbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 90.4 | — | — | |
| SafeChainbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 90.2 | — | — | |
| DS-7Bbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 89.8 | — | — | |
| SpecSamplingDraft Model=DeepseekR1-1.5B, Target Model=QwQ-32B2026.02 | 89.5 | 52.61 | 90.33 | |
| UnSafeChainbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 89.1 | — | — | |
| SpecSamplingDraft Model=DeepseekR1-1.5B, Target Model=DeepseekR1-32B2026.02 | 88 | 68.58 | 59.32 | |
| R2Dbase_model=DeepSeek-R1-Distill-Qwen-7B2026.03 | 84.4 | — | — | |
| SpecReasonDraft Model=DeepseekR1-1.5B, Target Model=DeepseekR1-32B2026.02 | 84 | 149.19 | 31.92 | |
| SpecThinkingDraft Model=DeepseekR1-1.5B, Target Model=DeepseekR1-32B2026.02 | 82.3 | 284.34 | 15.94 | |
| Draft ModelDraft Model=DeepseekR1-1.5B, Target Model=DeepseekR1-32B2026.02 | 81.42 | 380.51 | 12.7 | |
| Draft ModelDraft Model=DeepseekR1-1.5B, Target Model=QwQ-32B2026.02 | 81.42 | 380.51 | 12.7 | |
| SpecReasonDraft Model=DeepseekR1-1.5B, Target Model=QwQ-32B2026.02 | 77 | 76.02 | 53.66 | |
| SpecThinkingDraft Model=DeepseekR1-1.5B, Target Model=QwQ-32B2026.02 | 71 | 312.77 | 27.99 |