Code Generation on MBPP (Avg TPS, TPF, Latency, Score)
1TPFQwen3-8B
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen3-8BInference Engine / Platform=SGLang, Model Scale=8B2025.12 | 1 | 317.41 | 0.84 | 0.7892 | |
| SDAR-4B-ChatModel Scale=4B, Model Variant=Chat2025.12 | 1.5 | — | — | 0.52 | |
| SDAR-8B-ChatInference Engine / Platform=LMDeploy, Model Scale=8B, Model Variant=Chat2025.12 | 1.6 | 234.9 | 0.25 | 0.72 | |
| LLaDA2.0-flashInference Engine / Platform=SGLang, Model Variant=flash2025.12 | 2.7 | 433.81 | 0.21 | 0.883 | |
| D2F-Dream-7B-InstructInference Engine / Platform=CUDA, Model Scale=7B, Model Variant=Instruct2025.12 | 3.13 | 206.37 | 0.54 | 0.45 | |
| D2F-Dream-7B-BaseInference Engine / Platform=CUDA, Model Scale=7B, Model Variant=Base2025.12 | 5.64 | 327.69 | 1.91 | 0.45 |