Code Generation on DS1000
35AccuracySD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| SDModel=Qwen, Draft Model=0.5B, Target Model=14B2026.04 | 35 | 119 | — | — | |
| SMC-SDModel=Qwen, Draft Model=3B, Target Model=14B, Operating Point=iso-accuracy2026.04 | 33 | 244 | 2.05 | — | |
| SMC-SDModel=Qwen, Draft Model=3B, Target Model=14B, Operating Point=max speed2026.04 | 31 | 300 | 2.52 | — | |
| SDModel=Llama, Draft Model=1B, Target Model=8B2026.04 | 26.5 | 150 | — | — | |
| SMC-SDModel=Llama, Draft Model=1B, Target Model=8B, Operating Point=iso-accuracy2026.04 | 26.5 | 379.3 | 2.53 | — | |
| SMC-SDModel=Llama, Draft Model=1B, Target Model=8B, Operating Point=max speed2026.04 | 23.2 | 502.9 | 3.35 | — | |
| ckpt_-1SFT Status=Pre-SFT2026.05 | — | — | — | 28.6 | |
| MONAOptimizer=MONA, SFT Status=Post-SFT2026.05 | — | — | — | 37.03 | |
| MuonOptimizer=Muon, SFT Status=Post-SFT2026.05 | — | — | — | 33.95 |