Code Generation on HumanEval (TPS, Latency)
1,796.9TPS (Tokens/s)DFLARE
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| DFLAREConcurrency=322026.06 | 1,796.9 | — | — | — | |
| DFLAREConcurrency=162026.06 | 1,672.1 | — | — | — | |
| DFLAREConcurrency=82026.06 | 1,650.2 | — | — | — | |
| DFLAREConcurrency=42026.06 | 1,538.6 | — | — | — | |
| BaselineConcurrency=162026.06 | 1,273.7 | — | — | — | |
| BaselineConcurrency=322026.06 | 1,212 | — | — | — | |
| DFlashConcurrency=82026.06 | 1,178.7 | — | — | — | |
| DFlashConcurrency=162026.06 | 1,174.9 | — | — | — | |
| BaselineConcurrency=82026.06 | 1,158.2 | — | — | — | |
| DFlashConcurrency=322026.06 | 1,149.9 | — | — | — | |
| DFlashConcurrency=42026.06 | 1,142.9 | — | — | — | |
| BaselineConcurrency=42026.06 | 620.9 | — | — | — | |
| DFlashConcurrency=12026.06 | 561.1 | — | — | — | |
| DFLAREConcurrency=12026.06 | 554.5 | — | — | — | |
| BaselineConcurrency=12026.06 | 163.6 | — | — | — | |
| MoE-SpAc2026.02 | 28.24 | 16.93 | — | — | |
| llama.cppspeculative decoding=true2026.02 | 18.78 | 24.01 | — | — | |
| llama.cppspeculative decoding=false2026.02 | 14.74 | 30.57 | — | — | |
| HybriMoE2026.02 | 11.96 | 38.18 | — | — | |
| DDTree-oracleBackbone=Qwen3-4B, Temperature=0.02026.06 | 9.32 | — | 4.97 | 256 | |
| CADDTREEBackbone=Qwen3-4B, Temperature=0.02026.06 | 9.32 | — | 5.01 | 224 | |
| CADDTREEBackbone=Qwen3-8B, Temperature=0.02026.06 | 9.2 | — | 5.11 | 172 | |
| DDTree-oracleBackbone=Qwen3-8B, Temperature=0.02026.06 | 8.98 | — | 5.14 | 128 | |
| Fate2026.02 | 7.82 | 60.24 | — | — | |
| DFlashBackbone=Qwen3-4B, Temperature=0.02026.06 | 6.64 | — | 6.82 | — | |
| DFlashBackbone=Qwen3-8B, Temperature=0.02026.06 | 6.48 | — | 7 | — | |
| SP-MoE2026.02 | 4.98 | 86.93 | — | — | |
| MoE-Infinity2026.02 | 3.63 | 119.53 | — | — | |
| vLLM2026.02 | 2.63 | 158.46 | — | — | |
| Mixtral Offload2026.02 | 2.48 | 177.87 | — | — | |
| Accelerate2026.02 | 1.15 | 393.6 | — | — | |
| ARBackbone=Qwen3-4B, Temperature=0.02026.06 | — | — | 30.87 | — | |
| ARBackbone=Qwen3-8B, Temperature=0.02026.06 | — | — | 31.1 | — |