Code Generation on MBPP+ (Accuracy, TPS, Speedup)
170.9TPS (tok/s)PARSE + EAGLE3
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PARSE + EAGLE3Model=Qwen3-235B-A22B, Draft Model=Qwen3-8B, Decoding Strategy=PARSE with EAGLE3 on both draft and target, Precision=FP8, Inference Framework=SGLang2026.05 | 170.9 | — | 1.97 | — | |
| Eagle3Model=Qwen3-235B-A22B, Decoding Strategy=Speculative decoding, Precision=FP8, Inference Framework=SGLang2026.05 | 152.6 | — | 1.76 | — | |
| PARSEModel=Qwen3-235B-A22B, Draft Model=Qwen3-8B, Decoding Strategy=Full and partial verify, Precision=FP8, Inference Framework=SGLang2026.05 | 148.1 | 74.6 | 1.7 | — | |
| PARSETarget Model=GLM-4.7-FP8, Draft Model=Qwen3.5-9B2026.05 | 110.8 | 73.8 | 1.7 | — | |
| Qwen3-235B-A22BModel=Qwen3-235B-A22B, Decoding Strategy=Plain autoregressive, Precision=FP8, Inference Framework=SGLang2026.05 | 86.9 | 79.9 | — | — | |
| GLM-4.7-FP8Mode=Standalone2026.05 | 65.3 | 78 | — | — | |
| ckpt_-1SFT Status=Pre-SFT2026.05 | — | — | — | 0.5423 | |
| MONAOptimizer=MONA, SFT Status=Post-SFT2026.05 | — | — | — | 0.7857 | |
| MuonOptimizer=Muon, SFT Status=Post-SFT2026.05 | — | — | — | 0.7778 | |
| Qwen3-8BModel=Qwen3-8B, Decoding Strategy=Standalone2026.05 | — | 66.9 | — | — | |
| Qwen3.5-9BMode=Standalone2026.05 | — | 69.3 | — | — |